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ABSTRACT 



Previous work (Ginn, 1991) showed that faults in a program tend to 
cluster when viewed by the variables that affect execution and propagation 
of the fault (structural clustering). However, that study was quite preliminary 
and local in its investigation. This thesis examines clustering from two other 
perspectives, taxonomical (type of faults) and functional (area of affected 
functionality). The hypothesis tested was that faults tend to cluster when 
viewed from these perspectives. 

The approach was to use chi-square statistics on data taken from 
(Shimeall, 1991) to test the hypotheses, 247 faults were analyzed and the 
resultant clustering was cross-examined across the perspectives. 

The results show that the studied failure regions have a strong tendency 
to form taxonomical clusters. They also exhibit a mild tendency to form 
functional clusters. Taxonomical clustering does not correlate with structural 
clustering, w hile functional clustering correlates well with it. 
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I. INTRODUCTION 



A. MOTIVATION FOR THIS THESIS 

As computing systems perform more and more sophisticated functions, 
the software components of such systems necessarily become larger and 
more complex. In addition, a plethora of critical systems, such as nuclear 
power plants and aircraft control systems are controlled by software. The 
growth in size and complexity of software results in an even faster growth 
in the complexity of software testing. Testing is currently a very labor 
intensive and expensive process that accounts for approximately 50% of 
software system development (Myers 1979, Korel 1990). 

The need for reliable software operation is increasing rapidly. This 
implies that extensive software testing is frequently necessary despite 
expenses. This thesis deals with software testing, and is intended as a sequel 
to previous research on failure region identification (Shimeall et al 1991) and 
failure region clustering (Ginn, 1991). 
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B. OUTLINE OF THE PROBLEM 



Failure regions appear to have a tendency to cluster so the neighbo- 
rhood of a specific failure region may reveal more software faults than the 
one that caused the region. An empirical study (Ginn 1991) on data obtained 
by Shimeall and Leveson (1991) has demonstrated this clustering tendency 
of failure regions using appropriate clustering criteria for multidimensional 
nominal types of data (Jain and Dubes, 1988). A number of issues have been 
raised from this research: 



• Several failure regions demonstrated a strong clustering tendency in one 
dimension but weak clustering in other dimensions. However it is not 
known if this is a coincidence or normal behavior of failure regions. 

• Software faults were numbered in order as they were discovered, by the 
various testing techniques applied by Shimeall and Leveson (1991), so 
that many of the sequentially numbered faults were discovered by the 
same detection technique. There was also a strong tendency of 
clustering for sequential faults, but it is as yet unknown if there is a 
correlation between certain detection techniques and certain types of 
fault clusters. 

• It is not clear which types of conditions and variables are more likely 
to result in clusters. 
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There is empirical evidence that known failure regions may be used to 
understand the relationship of one fault to another. Failure regions offer a 
mechanism for identifying common features among faults, because the 
relationship between two failure regions corresponds to the relationship of 
the code locations of the associated faults. The goal of this research will be 
an improved testing technique that incorporates failure region behavior. To 
do this, we need to better understand how parts of the program interact, 
since faults with similarities in their failure regions are expected to occur 
under similar conditions. 

C. OVERVIEW OF THE THESIS 

Chapter II gives an extensive review of preceding research on software 
testing in general and failure region analysis in particular. Chapter III 
introduces a testing technique on clustering and presents empirical results 
from the application of the said technique on the results of a software 
experiment. Finally, Chapter IV summarizes the conclusions that can be 
drawn from the results and offers suggestions and recommendations for 



further research. 



II. BACKGROUND AND RELATED WORK 



This chapter reviews software testing definitions and methods for 
dealing with software fault and failure association. Symbols and terminology 
are similar to those used in the classical papers (such as Weyuker and 
Ostrand, 1980, Goodenough and Gerhart, 1975). A brief overview of theories 
of software testing related to our research is presented. Previous work, both 
empirical and theoretical, in the area of failure regions analysis is included. 
Finally the cluster analysis technique that has been used by Ginn (1991) is 
briefly presented. 

A. SOFTWARE TESTING 
1. Basic Definitions 

Testing is a method of program verification that deduces from 
execution that a program possesses required properties (Morell 1990). 

The input data for the majority of programs come from a multi- 
dimensional space. An example given by Amman and Knight (1988) refers 
to a program processing an input of 20 floating point numbers thus having 
20 dimensions. 
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We shall use D for the input domain of a program F, and R for the 
output range of F. On input d (deD) F, if it terminates, produces output 
F(d) (gR). Hence the program, may be viewed as a mapping from the input 
domain to the output range. 

The output specification for F is given by OUT(x,y), where xeD 
and ysR F is correct on input d (abbreviated OK(d)) if F(d) exists and 
OUT(d,F(d)). A test T (c D) is formally defined as a subset of the input 
domain (after Weyuker and Ostrand, 1980, Goodenough and Gerhart, 1975). 
More often than not, software failures appear at what seems to be an obscure 
set of data or a special case (Ammann and Knight 1988). This introduces an 
additional complication to the testing process. 

2. Ideal, Valid, and Reliable Tests 

An ideal test must be valid, which implies that for every fault in 
program F exist entries in test T that cause the fault to execute and produce 
incorrect output (Goodenough and Gerhart 1975). Formally: 

IdealiT) «- (V Fault e F: 3deTA^OK(d)) (2.1) 

A reliable test is one that either produces entirely correct output or 
entirely incorrect. Formally: 
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Re liable (T) -[(We T: OK{d)) V (Vde T: - OK(d ))] 



( 2 . 2 ) 



The importance of a failure region clustering theory becomes apparent 
here from the point of view of test selection. It was shown by Weyuker and 
Ostrand, (1980) that an ideal and reliable test is exhaustive therefore it is 
desirable, for the sake of feasibility, to select a much smaller test to the end 
of revealing certain types of faults. A failure region clustering theory 
provides same guidelines for this type of selection. 

B. PROGRAM PATHS 

1. The Concept of a Program Path 

A path in a program is defined as a sequence of statements (Rich- 
ardson and Clarke 1985). It is quite obvious that a path may be decomposed 
into a number of subsidiary paths. A block statement is decomposed into its 
constituent enclosed statements whose execution depends upon evaluation of 
a condition which is also part of the block statement. 

The control flow statements in a computer program partition the 
input space into a set of mutually exclusive domains each of which causes 
a corresponding path to be executed (White and Perera 1986). This concept 
of path offers a natural way to partition the input domain. 
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The subdomain Dj of path j, is defined by a boolean expression 
(say P(j)) that is the conjunction of the path's branch predicate constraints. 

In general these predicates are expressed in terms of both local (program) 
and input variables. However it is possible to replace each program variable 
appearing in the predicates by its symbolic value defined in terms of input 
variables along that path and get an equivalent constraint that is the partition 
boundary, P(j), as a function of input variables only (also predicate 
interpretation after White and Wiszienwski 1988). 

2. Programs as Sets of Partial Functions 

Using the concept of path, as defined in the previous subsection, 
we can model a program as a set of partial functions from the input 
partitions (Dj) to the output space (O), each of the partial functions 
corresponding to the execution of a sequence of statements along the 
corresponding path (j) (Richardson and Clarke 1985). Formally: 

S ul S ia ..SJD i ) = OtO (2.3) 

where Sj ,Sj 2 ..Sj n stands for the sequence of statements' along path j. In the 
case of loops executing along the path (let S j l S j l+l ..S j l+m be the sub sequence 

The condition evaluations are included into the sequence of statements for 
consistency although, since the path is pre- determined they do not affect 
the result of the operation. 
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of statements in the loop body) the path sequence of statements may be 
written Sj ,S j2 ( S j jS j , +I ..S j<1+m ) + ..S jn . This model assigns sequences that differ 
only by their number of loop iterations to the same program path. 

As a path may be decomposed into other paths, an input domain 
partition may be refined by the same token in a hierarchical way, as new 
branch predicate conditions are "AND ed" to the existing. This suggests a 
more generic approach to the issue of input domain partition than the one 
suggested by Richardson and Clarke (1985) where they distinguish this 
partition into implementation and specification partitions and point out 
certain discrepancies due to the inherent differences of the specification 
versus the implementation languages. 

3. Regular Expressions for Paths 

The set of paths, on a flow-chart, can be expressed in algebraic 
form (Beizer 1990). Path expressions are converted to regular expressions 
that can be used to examine structural properties of program paths. 

For any single path, j, of the program, the sequence of statements 
Sj ,Sj 2 ..Sj n introduced in the previous section is the path product of j. Path 
products are also defined on path segments. The path that consists of 
successive path segments, has a path product equal to the concatenation of 
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their path products. A set of parallel paths between two nodes has a path 
product equal to the sum of the path products of the parallel paths. 

Condition evaluations are retained in the path products for 
consistency (cf. previous subsection). In fact, the path products of the two 
segments starting from a decision point (such as an if C then else construct 
or a loop exit condition etc) are preceded with the condition C and condition 
iC respectively, depending upon which part of the decision (if or else) 
results in their execution. The regular expressions for paths provide a concise 
and compact notation for both the path conditions (which, for a given set of 
paths, are derived by conjunction of all conditions on successive path 
segments and disjunction of conditions on parallel path segments) and path 
actions (which, in the previous section, are modelled as partial functions 
from the input domain to the output range). 

C. ON ERRORS AND FAULTS 

A fault is an erroneous piece of program source code, while an error 
is a discrepancy between a computed value and the true, specified, or 
theoretically correct value. 



9 



A failure specifies the inability of a module to perform its specified 
function and includes both erroneous output and failure to produce output 
(see ANSI-IEEE STD 610.12-1990). 

A fault (E) is formally modeled as a 3-tuple (Shimeall et al 1991): 

E = <L„V„C > (2.4) 

where L r is the location of the fault 2 (which is some program statement), V r 
the list of variables that form the error caused by the fault and C r is a 
Boolean condition under which the fault is activated. An interesting point is 
that a fault activation does not always imply a failure (coincidental 
correctness. Morel 1988). There are three conditions that must hold true for 
a fault to produce a failure (Shimeall et al 1991) which, apart from C p are 
the reachability condition and the error propagation condition. 

A basic assumption about the faults in code is the competent program- 
mer hypothesis (DeMillo et al 1978). This states that a competent program- 
mer will write a program that is syntactically close to the correct program. 

A taxonomy of the types of faults possibly found in a program is given 
by Beiser (1990): 

Sometimes a fault is "distributed" to more than one locations while 
certain types of faults, such as missing functions etc., do not have a well 
defined location. 
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1. Requirements and Specifications 

These include incomplete or self-contradictory specifications, 
missing, wrong or superfluous features, and not-well-specified feature 
interactions. 

2. Structural Errors 

Such as control and sequence errors, logic errors, incorrect 
formulae applications, use of uninitialized variables. These can be further 
divided (Richardson and Clarke 1985) into: 

a. Computation Errors 

A computation error occurs when the correct path through the 
program is taken, but the output is incorrect because of faults in the 
computation along the path. 

b. Domain Errors 

A domain error (White and Perera 1986) occurs when a specific 
input follows the wrong path due to an error in the control flow of the 
program. They are of two kinds: 
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(1) Missing Path Errors. They occur when a special case 
requires a unique sequence of actions, but the program does not contain a 
corresponding path. 

(2) Path Selection Errors : They occur when the program 

recognizes the need for a path, but incorrectly determines the conditions 
under which the path is executed. 

3. Data Errore 

Occur when a specific input follows the correct path, but an error 
such as wrong data declarations, wrong data initialization (especially in 
shared dynamic objects), etc. results in erroneous output. 

4. Coding Errors 

The most common coding errors are documentation inconsistencies, 
typographical errors, and erroneous use of a program statement when its side 
effects are not well understood. 

5. Interface Errors 

Such as interface communication problems, incorrect input-output 
format, wrong subroutine control sequence, wrong call parameters, incon- 
sistent entry or exit parameter values, etc. 
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D. ON FAILURE REGIONS 



A software failure region (GeD) is the set of all input values that are 
mapped by an individual program fault onto any failure (or onto a failure set, 
as in figure 1.1). It is noted that the concept of failure region includes both 
the input points which cause erroneous output and the geometry of it. These 
sets are always finite, since the number of representations in a finite machine 
is limited, but more often than not intractably large. The failure region boun- 
daries are defined by Boolean conditions on the input domain. 

Shimeall et al (1991) demonstrate a technique that analytically de- 
termines the three conditions for a known fault (which include all conditions 
for reaching, activating and propagating the fault), by symbolically executing 
(King 1976) the source code on every "loop [0,1]" path (Loops are handled 
as in previous section, by applying either the exit condition or the loop 
effects) and conjuncting the obtained Boolean expressions. The conjunction 
of those conditions is the mathematical specification of the failure region 
boundary (which may be called Bound(G)) subject to the limitations of finite 
representation in computing machines. 
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Figure 1.1 Associations between Failure regions Faults 
and Failures (adapted from Shimeall et al 
1991) 

E. BOUNDS CORRELATION BETWEEN DIFFERENT FAILURE 
REGIONS 

The dimensions of the input domain of a program can provide a set of 
criteria for the correlation between different failure regions. To this end a 
classification of these dimensions is used (Ginn 1991), with respect to any 
pair of failure regions: 
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• The dimensions that appear in both boundaries in exactly the same way 
are termed identically participating dimensions. 

• Those who appear in both boundaries but not in an identical way are 
termed coincidentally participating dimensions. 

• The dimensions that do not appear in the boundaries of both regions are 
called nonbounding dimensions, because the boundaries these dim- 
ensions place on failure regions are no more restrictive than their entire 
range of values. 



In Ginn (1991) the Identical and Coincidental dimensions are collective- 
ly referred to as Composite dimensions. 
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III. CLUSTER ANALYSIS AS A PREDICTOR FOR FAULTS 



A. CLUSTER ANALYSIS 

The identification of groups that have similar characteristics is the goal 
of clustering analysis. In the context of this thesis, the objective of failure 
region analysis is to investigate clustering of failure regions and identify the 
logical relation of the program locations of the faults responsible for these 
failure regions. 

By definition of failure region (Shimeall et al 1991 for example) the 
execution of a program, with data from some failure region as input, will 
reach , activate, and propagate to the output the corresponding program fault. 
The conjunction of predicates that defines the bound of the region (Chapter 
II-D) contains a statement of the reachability, activation and propagation 
conditions for the corresponding fault. 

The criterion used by Ginn (1991) assumed that clustering between 
failure regions occurs when identical or similar predicates appear in the 
bounds of these failure regions. Therefore, one could use identical dim- 
ensions (corresponding to identical predicates) or composite dimensions 
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(corresponding to both identical and similar predicates) to define a measure 
of clustering. The assumption made by Ginn (1991) is reasonable, because 
it is expected that when failure regions correspond to faults that share some 
program paths (therefore some of the predicates in common) it is possible, 
but not necessary, that the same input may reveal all of them. On the 
contrary, when the faults do not share any path, it is impossible for both of 
them to be revealed by the same input. 

The criteria used for the failure region bounds variables are essentially 
ordinal: identical, coincidental, nonbounding, in descending order. Therefore 
relative values cannot be assigned to them. There are, however, two different 
coefficients that may serve as a measure of clustering for ordinal data (see 
Jain and Dubes 1988): 






a oo +a u 

a 



( 3 . 1 ) 



The simple matching coefficients for two failure regions G s , Gj are 
defined by 3.1, where stands for the non-bounding dimensions of both 
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failure regions a,, for the shared 3 dimensions of Gi and Gj, (cf. Chapter II-E) 
and a for the total number of input dimensions. The similarity of failure 
regions G s and Gj is greater the closer the simple matching coefficient is to 
unity, since S(Gj,G;)=l, because a^a^M). The inclusion of a^ in the 
numerator of the coefficient emphasizes equally bounding and non-bounding 
dimensions and can result in very high (close to unity) values of the 
coefficient when the total number of input dimensions is much greater than 
the bounding dimensions of two failure regions. Suppose for example the 
pair of failure regions: 

Bounds G x ) = {x <5) A (y < 8) A (z < 10) (3.2) 

Bound(G 2 ) = (w<0)A(w<0) (3.3) 

If the total number of dimensions is 250 then 300=245, a n =0, and 
S(G„G 2 )=245/250=0.98, which implies a high degree of clustering while it 
is obvious that the failure regions are not related. 



The term shared may refer to dimensions appearing in identical predicates in two 
failure regions definitions, or may include both similar and identical predicates. Ginn 
(1991) investigates both cases separately. 
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The Jaccard coefficients, defined by 3.4, seem more sensitive to 
clustering since they emphasize more the composite dimensions, as opposed 
to the simple matching coefficients that are symmetric in composite an non- 
coincidental dimensions. As with the simple matching coefficients, the 
similarity of failure regions G s and Gj is greater the closer the Jacard 
coefficient is to unity, since J(Gj,Gj)=l. 

J(G,,Gj) = (3.4) 

a - a oo 

However, under certain circumstances, the reliability of Jaccard 
coefficients is debatable. Suppose for example the pair of failure regions 
defined by (3.5), (3.6): 

Bound(G t ) = (x<5)A(y<8)A(z< 10) (3.5) 

Bound(G 2 ) = (jc<5)A(y<8)A(z<10)A(w<0)A(w<0) (3.6) 



19 



In this case the Jacard coefficient J(G„G 2 )=3/(3+2+0)=0.6, while it is 
intuitively obvious that the relation is probably stronger than is suggested by 
the coefficient. On the other hand, the simple matching coefficient, 
depending upon the number of dimensions that are non-bounding for both Gj 
and G 2 may vary between 0.6 and 0.9999. 

In the following sections, we shall examine how pairs of faults may 
result in clustered failure regions, as well as how the Jaccard coefficients are 
affected. 

B. RELATED FAULTS 

The goal in analyzing fault relations is to understand the clustered 
failure regions, which appear in software experiments (Ginn 1991). To this 
end, it is required to determine what kind of fault relationship results in 
clustered failure regions. 

L Logical Relation of Faults 

Faults can be logically related if they are either the same logical 
flaw ( Taxononucally related) or they are located in regions of programs that 
compute the same part of the application ( Functionally Related). 
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2. TaxonomicaJly Related Faults 



A logical flaw is an error, or misunderstanding by the programmer, 
in the design logic of a program that results in a number of faults. Taxo- 
nomically related faults are expected to be both application dependent and 
programmer dependent. They are application dependent because it is 
expected that applications requiring a great number of a certain type of 
constructs (say loops or if statements for example) are more prone to the 
type of faults peculiar to these constructs (loop iteration and control flow 
faults respectively for this example). It is not always clear from the 
specification of a program exactly how many such constructs will be 
required, and it is usually up to the programmer to decide for the im- 
plementation details. However, the higher level design of a program always 
gives an indication whether the implementation requires many iterations or 
a lot of case handling etc. Therefore this type of faults depends both on the 
design and the implementation details. 

On the other hand they are programmer dependent because every 
programmer has his/her own weak and strong points in developing a 
software design, and it is obvious that some phases of his/her work will be 
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more prone to faults than others. This also holds true for both the design 
and the implementation phases. 

3. Taxonomical Fault Classification of a Software Experiment 

The classification of the known program faults in a set of eight 
redundant versions of a combat simulation program {Shimeall 1991a, 1991b) 
constructed as part of a software experiment (Shimeall and Leveson 1991) 
were analyzed. The fault categories used in this taxonomy are from Beiser 
(1990). According to his classification scheme each fault is characterized by 
the type of logical flaw that produced it (for example case selection bug, 
control logic bug etc) and is assigned a four digit code number. 

The first digit is characteristic of the highest level of the taxonomy 
hierarchy (i.e.. Structural Bug, has code 3xxx) while the last digit specifies 
the exact category of the fault (for example a Structural, control state fault, 
has code 3154). The advantages of this taxonomy is that provides an easy 
hierarchical and logical scheme, and the four digits of the code specify the 
four levels of the used hierarchy (cf. Chapter II section C for a discussion 
of the highest levels). 

The results of the fault classification in the eight versions are 
presented in Table 3.1 (detailed results including fault relative frequency 
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histograms for the eight versions, and for the observed to expected relative 
frequency ratio of the total number of faults are presented in Appendix B, 
Figures B.l to B.8). In addition to the fault statistics, in the table we include 
the number of statements, if and case selection statements, and loop 
constructs. 

4. The 'Taxonomica! Clustering” Hypothesis 

The testing of the "taxonondcal clustering" hypothesis is 
performed by comparison of the actual results with a simple random model 
that assumes that, given the total number of faults in a program, the 
probability of an fault occurrence in any line is equal to the ratio of the total 
number of faults by the total number of lines. The, ratio, is the expected fault 
rate for the given program. The expected number of faults for each of the 
categories in Table 3.1, for the random fault distribution, is calculated as the 
product of the fault rate time the number of statements where this type of 
fault can occur. 

Within the scope of our working model, processing, initialization 
and algorithmic faults (codes 32 lx, 322x, 323x) occur in all program 
statements apart from if, case and loop constructs. Loop and iteration faults 
(code 314x) occur in loop constructs. Control logic, case selection and 
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control state faults (code 312x, 313x, 315x) may appear in if or case 
statements. The exception handling faults appear usually either in control (if, 
case) statements or loop exit conditions. Therefore we assume further that, 
on the average, half of the faults in loop and control statements are on 
exception handling and the other half on loop, iteration and control 
respectively. 

The random fault allocation that results from the use of our model, 
is also included in Table 3.1 for the eight program versions. In this case the 
"random" version number has the subscript r. 

In Appendix B we present the relative frequency distribution for 
both the actual number of faults and the model predictions. To test the 
goodness of fit of the observations to the random fault distribution model we 
make use of the Chi-Square (% 2 ) One-Sample-Test (Siegel and Castellan 
1988). This test gives the level of significance (probability of occurrence in 
fact) that the x 2 statistic, which increases with the difference between the 
observed and the expected fault distribution, is greater than a certain value. 
The higher the probability of the said difference the more confident we are 
that the selected model corresponds to the actual distribution. However, the 
results of Table 3.1 imply that the probability of the observed differences 
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between the actual number of faults and the model prediction is below 
0.001, because for 3 degrees of freedom (since there are four fault cate- 
gories) the smallest value of y} was 19.7 in version VI. Therefore we can 
conclude that the fault distribution is not uniform. This implies that the faults 
that have the most opportunity of being committed are not always the most 
frequent in a program. 

It is evident from the statistics in Table 3.1 that control logic, case 
selection and control state faults appear at a rate about fourteen times the 
average fault ratio in the program while processing and initialization fault 
rate is about 30% of the average. This leads to the conclusion that the 
handling of the control logic of the program, at least in the CONFLICT 
experiment (Shimeall 1991a, 1991b, Shimeall and Leveson 1991) and for the 
programmers selected, represents a task of much greater difficulty than 
processing and initialization in the same program. 
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TABLE 3.1 

(PROGRAM STATISTICS, FOR TAXONOMICAL CLUSTERING) 



Vers- 1 
ion 


if + 
case 
statem- 
ents 


loop 

const- 

ructs 


other 

statem- 

ents 


314x 
(loop 
and ite- 
ration) 


312x, 
313i, 
315x, 
(control- 
logic, 
case sele- 


321x322 

X 

3 23 1 , (pro ! 
cessing 
and in- 
itializa- 


Fault in 
excep- 
tion han- 
dling 
(316x) 


Total 












ction, 

control 

state) 


tion) 






I 


250 


184 


1980 


0 


9 


15 


2 


26 


K 


250 


184 


1980 


1 


1.3 


21.3 


2.3 


26 


2 

X 


- 


- 


- 


1 


45.6 


1.86 


0.04 


48.5 


□ 


123 


86 


1331 


0 


21 


3 


2 


26 


Hr 


123 


86 


1331 


0.73 


1.0 


22.5 


1.73 


26 


x 2 


- 


- 


- 


0.73 


400 


16.9 


0.04 


417.7 


in 


108 


88 


1005 


1 


27 


8 


3 


40 


HU 


108 


88 


1005 


1.43 


1.76 


32.2 


3.2 


39 


x 2 


- 


- 


- 


0.13 


362.0 


18.2 


0.01 


380.3 


IV 


206 


119 


1678 


1 


8 


7 


3 


23 


IV r 


206 


119 


1678 


0.6 


1 


16 


1.6 


19 


1 


- 


- 


- 


0.27 


49.0 


5.1 


1.2 


55.6 
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TABLE 3.1 

(PROGRAM STATISTICS, FOR TAXONOM1CAL CLUSTERING) 



Vers- 

ion 


if + 
case 
statem- 
ents 


loop 

const- 

ructs 


other 

statem- 

ents 


314x 
(loop 
and ite- 
ration) 


312x, 
3l3x, 
315x, 
(control- 
logic, 
case sele- 


321x, 
322x 
323x, 
(proc- 
essing 
and in- 


Fault in 
excep- 
tion han- 
dling 
(316x) 


Total 












ction, 

control 

state) 


itializa- 

tion) 






V 


153 


93 


1298 


0 


28 


6 


i 


40 


v r 


153 


93 


1298 


1 


1.7 


29 


2.7 


35 


2 

X 


- 


- 


- 


1 


406.9 


18.2 


1.1 


427.2 


VI 


200 


101 


1905 


1 


5 


13 


3 


22 


VI, 


200 


101 


1905 


0.55 


1.1 


19.02 


1.32 


22 


x 2 


- 


- 


- 


0.37 


13.8 


3.36 


2.14 


18.2 


vn 


201 


133 


1644 


1 


18 


8 


3 


31 


vn . 


201 


133 


1644 


1 


1.5 


25 


2.5 


30 


2 

X 


- 


- 


- 


0 


181.5 


11.6 


0.1 


193.2 


VIII 


95 


78 


1158 


1 


31 


1 


8 


41 


VIII r 


95 


78 


1158 


1.2 


1.5 


36.5 


2.7 


41 


2 

X 


- 


- 


- 


0.04 


580.2 


34.5 


10.4 


639 


TOTAL 


1336 


882 


11999 


5 


148 


61 


25 


248 


TOTA- 

Lr 


1336 


882 


11999 


7.51 


10.8 


209.4 


18.2 


239 


2 

X 


- 


- 


- 


0.8 


1741.0 


105.4 


2.6 


1843 
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5. Taxonomical Clustering of Failure Regions Compared to the 

Structural Clustering of Failure Regions 

In this section, we examine whether the logical clustering of failure 
regions correlates with the number of shared dimensions in their bounds. The 
number of shared dimensions at the bounds of two failure regions has been 
used in the definition of a clustering metric, the Jaccard coefficient between 
two regions, by Ginn (1991). On the other hand, we consider two failure 
regions as members of the same Taxonomical cluster if the associated faults 
are both results of similar logical flaws. We classify the Taxonomical 
clusters as follows: 

• type A. Corresponds to loop and iteration faults (code 314x). 

• type B. Corresponds to control logic, case selection and control state 
(codes 312x, 313x, 315x) faults. 

• type C: Corresponds to processing, initialization and algorithmic faults 
(codes 32 lx, 322x, 323x). 

• type D: Corresponds to faults in exception handling (code 316x). 

Summary statistics of this comparison are presented in Table 3.2 
(Details are included in Appendix D). From these results it is apparent that 
the appearance of shared dimensions, and the variation of the Jaccard 
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coefficient, on failure region bounds does not depend on the taxonomical 
clustering as analyzed in subsection b, where the clustering criterion between 
different failure regions has been whether they correspond to faults resulting 
from the same logical flaw. 



TABLE 3.2 

AVERAGE AND STANDARD DEVIATION OF THE JACCARD 
COEFFICIENT BETWEEN TAXONOMICALLY CORRELATED 

FAILURE REGIONS 



VER- 

SION 


Failure 
region pair 
of type A 


Failure region 
pair of type B 


Failure region 
pair of type C 


Failure region 
pair of type D 


Uncorrelated 
failure region pair 


1 


- 


0.05 1±0. 115 


0.04910.134 


0.25 4 


0.04210.108 


n 


- 


0.050±0.149 


0.133 


- 


0.02910.066 


m 


- 


0.076±0.135 


0.05410.107 


0 


0.05510.115 


IV 


- 


0.032±0.061 


0.05010.118 


0.077 


0.03610.092 


V 


- 


0.019±0.085 


0.01710.034 


- 


0.00810.035 


VI 


- 


0.073+0.230 


0.05910.118 


- 


0.01810.039 


vn 


- 


0.07410.174 


0.03110.075 


0.11110.19 


0.01810.065 


VIII 


- 


0.12310.429 


- 


0.13410.181 


0.11710.227 



In Table 3.3 we present the same statistics as in Table 3.2 with 
the difference that, in this case, we use the average non-zero Jaccard 
coefficient between taxonomically correlated failure region. This indicates 

In entries without Standard Deviation there is only one value. 
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whether the taxonomical clustering of failure regions correlates with the 
number of shared dimensions in their bounds, when these shared dimensions 
exist. 



TABLE 3.3 

AVERAGE AND STANDARD DEVIATION OF THE NON ZERO 
JACCARD COEFFICIENT BETWEEN TAXONOMICALLY 
CORRELATED FAILURE REGIONS 



VER- 

SION 


Failure 
region pair 
of type A 


Failure region 
pair of type B 


Failure region 
pair of type C 


Failure region 
pair of type D 


II 

Uncorrelated 
failure region pair 


I 


- 


0.124±0.150 


0.24710.361 


0.250 


0.16410.159 


n 


- 


0.393±0. 196 


0.133 


- 


0.12810.09 


m 


- 


0. 1 7 1 ±0. 1 39 


0.13910.134 


- 


0.15910.143 


IV 


- 


0.129±0.048 


0.15610.167 


0.077 


0.13110.136 


V 


- 


0.31510.172 


0.067 


- 


0.11510.089 


VI 


- 


0.73 


0.19110.147 


- 


0.08610.026 


vn 


- 


0.43410.147 


0.13310.109 


0.333 


0.17310.110 


Vlll 


- 


0.46910.222 


- 


0.28310.117 


0.38710.258 



The results in Table 3.3 provide us with evidence in favor of the 
intuitively obvious hypothesis that the structural clustering of failure regions 
does not depend on the taxonomical clustering. This result can be attributed 
to the fact that structural clustering of failure regions, depends strongly on 
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the control and data flow structure of the program, with respect to the 
corresponding fault location. On the other hand the taxonomical clustering 
dependence is restricted to the type of statement at the fault location. 

6. The Case of Functionally Related Faults 

The hypothesis, that logically-related faults located in regions of 
programs that compute the same part of the application may lead to some 
type of clustering, is based on the intuitively obvious assumption that some 
parts of a problem may be more difficult to handle or more "error prone" 
than others (Brilliant et al 1990). We shall call this type of faults functio- 
nally related. 

7. Functional Fault Classification of a Software Experiment 

The distribution of the known program faults in the eight versions 
of the combat simulation program (Shimeall 1991, 1991b) to the program 
modules implementing different functional requirements of the specification 
was analyzed. We classified the functional clusters, according to the 
CONFLICT Specification (Shimeall 1991b) as follows: 

• Type /: Positioning and Movement 
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• Type II: Observation 

• Type HI: Attrition 

• Type IV: Communication 

• Type V: Environment 

• Type VI: Restoration 

• Type O: Others (Includes the main procedure of CONFLICT and some 
procedures for initialization and output format) 

8. The Functional Clustering Hypothesis 

The null hypothesis that fault distribution is uniform over the 
program locations was used in this analysis to test the clustering hypothesis. 
The number of expected faults for each functional requirement of the 
specification was set equal to the total number of lines of routines im- 
plementing the requirement times the fault rate (as in subsection lb) for the 
program. 

The results of this analysis are presented in Table 3.4., while 
detailed analysis is included in Appendix D. 

To test the goodness of fit of the observations to the random fault 
distribution model we make use of the Chi-Square (% 2 ) One-Sample-Test 
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(Siegel and Castellan 1988), as in subsection 4. In this case there are seven 
categories of faults, therefore the degrees of freedom are six. 

Apart from versions IV, VII and VIII, the confidence level for the 
null hypothesis is varying from 1% to 31%. In versions IV, VII and VIII this 
confidence drops below 0.1%. This result, however, may be attributed to the 
low (less than five) expected number of type I, II, III, V, VI and O faults for 
version IV and the low expected number of type I, V, VI faults for version 
VIII. In version VII the result may be attributed to the low, 3.1, number of 
expected fault at column IV compared to the 10.5 observed. 

The above discussion can be verified in case we consider only two 
fault categories for version IV, Type IV with 7 observed and 6.7 expected 
faults, and all others with 16 observed and 16.3 expected faults. This will 
result in a x 2 equal to 0.019 and a corresponding confidence level, one 
degree of freedom this time, greater than 90%. 

In the case of version VIII, we may consider the fault categories 
II, III, IV, O and all others, with four degrees of freedom, and the x 2 will be 
equal to 14.85, which gives a confidence level of 1%. 

The confidence level increases accordingly with all the remaining 
versions if we group together all categories with expectation value less than 
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5 as suggested by Siegel and Castellan (1988). Under the circumstances, we 
cannot either accept or reject the null hypothesis of uniform distribution of 
faults. 

An alternate hypothesis, that the faults are uniformly distributed 
between type IV (Communication), type III+II (Attrition and Observation) 
and all other types is tested in Table 3.5. The alternate hypothesis assumes 
that some type of functional clustering exists, because 2/3 of the faults are 
clustered in the Communication, Attrition and Observation modules, which, 
on the average, constitute the 50% of the total lines of code. 



TABLE 3.4 

(PROGRAM STATISTICS, FOR FUNCTIONAL CLUSTERING) 



Version 


Type I 


Type 

n 


Type 

in 


Type 

IV 


Type 

V 


Type 

VI 


Type 

O 


TO- 

TAL 


Confidence 

Level 


i 


1 


3 


4 


3 


4 


3 


8 


26 


- 


K 


1.77 


2.65 


3.92 


7.49 


1.58 


2.38 


6.2 ! 


26 1 


- 


x 2 


0.33 


0.05 


0.002 


2.7 


3.71 


0.16 


0.52 


7.5 


28.4% 


n 


3 


5 


0 


14 


3 


1 


0 


26 


- 


Hr 


4.10 


3.83 


4.85 


6.23 


2.38 


1.5 


3.11 


26 


- 


x 2 


0.295 


0.008 


4.85 


9.7 


0.16 


0.17 


3.11 


18.3 


1% 
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TABLE 3.4 (PROGRAM STATISTICS, FOR FUNCTIONAL CLUSTER- 
ING) 



Vernon 


Type I 


Type D 


Type III 


Type IV 


Type V 


Type 

VI 


Type 

O 


TO- 

TAL 


Confide nee 
Level 


m 


2 


3 


7 


13.5 


5 


2 


7.5 


40 


- 


nu 


3.13 


4.9 


4.37 


13.57 


2.83 


0.8 


10.4 


40 


- 


x 2 


0.41 


0.74 


1.6 


0.00 


1.66 


1.8 


0.81 


7.0 


31% 


IV 


0 


11.5 


2.5 


7 


1 


1 


0 


23 


- 


rv r 


2.4 


3.07 


3.7 


6.7 


1.7 


2.3 


3.1 


23 


- 


2 

X 


2.4 


23.15 


0.39 


0.013 


0.29 


0.04 


3.1 


29.4 


<0.1% 


V 


0 


6 


2 


15.5 


1.5 


2 


13 


40 


- 


V r 


2.6 


7.9 


7.3 


9.0 


2.3 


1.5 


9.4 


40 


- 


2 

X 


2.6 


0.60 


3.85 


4.69 


0.28 


0.17 


1.38 


13.5 


5% 


VI 


0 


3 


5 


7.5 


0 


0 


4.5 


20 


- 


vl 


0.87 


2.9 


1.8 


8.1 


1.6 


1.03 


3.7 


20 


- 


2 

X 


0.87 


0.00 


5.7 


0.04 


1.6 


1.03 


0.17 


9.4 


25% 


vn 


0 


7 


5 


10.5 


2 


2 


4.5 


31 


- 


Vn r 


5.0 


4.6 


3.8 


3.1 


2.4 


2.0 


10.4 


31 


- 


2 

X 


5.0 


1.25 


0.38 


17.7 


0.07 


0 


3.35 


27.8 


<0.1% 


VIII 


4 


11 


3 


15 


7 


1 


0 


41 


- 


VIIl r 


4.2 


5.9 


6.4 


10.8 


3.3 


2.7 


7.8 


41 


- 


x 2 


0.009 


4.41 


1.8 


1.63 


4.15 


1.1 


7.8 


21.0 


0.1% 


TOTAL 


10 


49.5 


28.5 


86 


23.5 


12 


37.5 


247 


- 


TOTAL* 


24.1 


35.6 


36.1 


65 


18.1 


14.23 


62.8 


247 


- 


x 2 


22.7 


5.43 


1.6 


6.78 


1.61 


0.35 


10.2 


48.6 


<0.01% 
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TABLE 3.5 (ALTERNATE HYPOTHESIS TEST) 



Version 


Type IV 


Type I1+1II 


Type 

O+I+V+VI 


TOTAL 


Confidence 

Level 


i 


3 


7 


16 


26 


- 


4 


8.67 


8.67 


8.67 


26 


- 


x 2 


3.71 


0.32 


6.2 


10.23 


0.6% 


II 


14 


5 


7 


26 


- 


E4 


8.67 


8.67 


8.67 


26 


- 


2 

X 


3.28 


1.55 


0.32 


5.15 


7.6% 


in 


13.5 


10 


16.5 


40 


- 




13.33 


13.33 


13.33 


40 


- 


x 2 


0.00 


0.833 


0.752 


1.59 


92% 


IV 


7 


14 


2 


23 


- 


IV r 


7.67 


7.67 


7.67 


23 


- 


x 2 


0.06 


5.22 


4.19 


9.48 


0.9% 


V 


15.5 


8 


16.5 


40 


- 


V r 


13.33 


13.33 


13.33 


40 


- 


2 

X 


0.35 


2.13 


0.75 


3.23 


20% 


VI 


7.5 


8 


4.5 


20 


- 


V4 


6.67 


6.67 


6.67 


20 


- 


x 2 


0.10 


0.27 


0.71 


1.08 


60% 


vn 


10.5 


12 


8.5 


31 


- 


V 14 


10.33 


10.33 


10.33 


31 


- 


2 

X 


0.00 


0.27 


0.33 


0.60 


75% 
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TABLE 3.5 (ALTERNATE HYPOTHESIS TEST) 



Version 


Type IV 


Type II+III 


Type 

O+I+V+VI 


TOTAL 


Confidence Level 


VIII 


15 


14 


12 


41 




vm r 


13.67 


13.67 


13.67 


41 


- 


X 


0.13 


0.01 


0.20 


0.34 


84.3% 


TOTAL 


86 


78 


83 


247 


- 


TOTALr 


82.33 


82.33 


82.33 


247 


- 


2 

X 


0.16 


0.23 


0.01 


0.40 


82% 



The confidence levels of the chi-square test range from 84.3% to 
0.6%. Although we cannot accept or reject the alternate hypothesis, we 
notice that the confidence levels for acceptance are in general higher that 
those of the null hypothesis. The results for versions III, VI, VII and VIII are 
in favor of the alternate hypothesis, which implies some type of functional 
clustering. The results for versions II, IV, V are not in favor of the alternate 
hypothesis because the functional clustering tendency is stronger than the 
assumed by the alternate hypothesis, and only version I implies that there is 
not any clustering tendency favoring Attrition Communication and Observa- 
tion. 
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Therefore it is reasonable to conclude from the data at hand that there is 



indeed some, not very strong, tendency of the failure regions to cluster in 
some groups of functional program modules more than in others. 

9. Clustering of Failure Regions of Functionally Related Faults Com- 
pared to the Structural Clustering of Failure Regions 

In this section, we examine whether the functional clustering of 
failure regions correlates with the number of shared dimensions in their 
bounds. In this case we consider two failure regions as members of the 
same functional cluster if the associated faults appear in regions of the 
program that compute the same part of the application. Summary statistics 
of this comparison are presented in Table 3.6 and 3.7 (the latter corresponds 
to the non zero Jaccard coefficient case) while details are included in 
Appendix D. 

From the results of Table 3.6 it is not possible to conclude that the 
appearance of shared dimensions, and the variation of the Jaccard coefficient, 
on failure region bounds does or does not depend on the functional 
clustering, as defined in subsection 8, because the calculated standard 
deviations exceed the averages due to the abundance of zero data values. 
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However, from Table 3.7, we can see that the average non zero 
Jaccard coefficient is systematically higher for functionally correlated pairs 
that for uncorrelated. This implies that, given the structural clustering of two 
failure regions, the clustering metric is higher for functionally correlated 
ones. This result can be attributed to the fact that structural clustering of 
failure regions depends on the control and data flow structure of the 
program, with respect to the corresponding fault location. 

It is expected that in a reasonably well structured program, faults 
on the same group of functional modules will share a common control and 
data flow path segment more often than faults on different groups. 
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TABLE 3.6 

AVERAGE AND STANDARD DEVIATION OF THE JACCARD 
COEFFICIENT BETWEEN FUNCTIONALLY CORRELATED 

FAILURE REGIONS 6 



VERSION 


Failure 
Region 
Pair of 
Type 1 


Failure 
Region 
Pair of 
Type II 


Failure 
Region 
Pair of 
Type ID 


Failure 
Region 
Pair of 
Type IV 


Failure 
Region 
Pair of 
Type V 


Failure 
Region 
Pair of 
Type VI 


Failure 
Region 
Pair of 
Type O 


Failure 

Region 

Pair 

Uncor- 

related 


1 


- 


0.025 

±0.04 


0.140 

±0.006 


0.042 


0.240 

±0.49 


0.125 

±0.21 


0.096 

±0 .K 


0.0284 

±0.081 


n 


0±0 


0.091 

±0.24 


- 


0.076 

±0.184 


0.111 

±0.192 




- 


0.0181 

±0.073 


m 


0.250 


0±0 


0.223 

±0.312 


0.097 

±0.137 


0.148 

±0.091 


0 


0.096 

±0.14 


0.046 

±0.095 


IV 


- 


0.0374 

±0.064 


0.056 

±0.096 


0.0823 

±0.235 


- 


- 


- 


0.019 

±0.053 


V 


- 


0±0 


0.714 


0.0227 

±0.086 


0 




0.005 

±0.03 


0.0087 

±0.051 


VI 


- 


0.336 

±0.043 


0.1142 

±0.220 


0.075 

±0.127 


- 




0.0643 

±0.16 


0.0184 

±0.071 


VII 


- 


0.0263 

±0.058 


0.2922 

±0.239 


0.131 

±0.220 


0.333 


0 


0.117 

±0.19 


0.0192 

±0.071 


vm 


0.125 

±0.25 


0.333 

±0.315 


0.75 


0.102 

±0.206 


0.187 

±0.285 


- 


- 


0.069 

±0.180 



In entries without Standard Deviation there is only one value. 
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TABLE 3.7 

AVERAGE AND STANDARD DEVIATION OF THE NON ZERO 
JACCARD COEFFICIENT BETWEEN FUNCTIONALLY 
CORRELATED FAILURE REGIONS 7 



VERSION 


Failure 
Region 
Pair of 
Type I 


Failure 
Region 
Pair of 
Type II 


Failure 
Region 
Pair of 
Type ID 


Failure 
Region 
Pair of 
Type IV 


Failure 
Region 
Pair of 
Type V 


Failure 
Region 
Pair of 
Type VI 


Failure 
Region 
Pair of 
Type O 


Failure 

Region 

Pair 

Uncor- 

related 


i 


“ 


0.074 


0 . 140 ± 

0.0060 


0.042 


0 . 795 ± 


0 t 3 - 50 ± 


0.430 

± 

0.110 


0.121 

± 0.131 


n 


- 


0.636 


- 


0 . 407 ± 

0.216 


0.333 


- 


- 


0 . 206 ± 

0.152 


in 


0.250 


- 


0 . 586 ± 

0.189 


0 . 244 ± 

0.168 


0 . 1 48 ± 
0.091 


- 


0.240 

± 

0.149 


0 . 145 ± 

0.116 


IV 


- 


0 . 1 27 ± 
0.053 


0.167 


0.70 


- 


- 


- 


0 . 1 03 ± 
0.080 


V 


- 


- 




0 . 244 ± 

0.160 


- 


- 


0.170 

± 

0.029 


0 . 1 70 ± 
0.177 


VI 


- 


0 . 336 ± 

0.043 


0 . 1 90 ± 
0.265 


0 . 1 86 ± 
0.156 


- 


- 


0.50 


0 . 1 00 ± 
0.070 


vn 


- 


0 . 106 ± 

0.073 


0.2922 

± 0.239 


0 . 476 ± 

0.088 


0.333 


- 


0.389 

± 

0.096 


0 . 1 96 ± 
0.130 


Mil 

L 


0.500 


0 . 438 ± 

0.285 


0.75 


0 . 442 ± 

0.182 


0 . 427 ± 

0.269 


- 


- 


0 . 394 ± 
0.237 j 



In entries without Standard Deviation there is only one value. Crossed out 
entries, contain Average and Standard deviation of only two data values 
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C. CONCLUSIONS ON CHAPTER ffl 



In this chapter the failure region analysis on the results of the 
CONFLICT experiment identifies the logical relation of the faults responsible 
for the observed failure regions. 

The testing of the taxonomical clustering hypothesis, section 3.3 to 3.5, 
implies the observed fault distribution is not uniform. The control logic, case 
selection, and control state faults appear at a rate about fourteen times the 
average fault ratio in the program while processing and initialization fault 
rate is about 30% of the average. Therefore control logic faults and 
corresponding failure regions actually exhibit taxonomical clustering 
behavior. This, however, does not correlate with the structural clustering of 
failure regions observed by Ginn (1991) on the same set of data. This can 
be justified by the fact that the latter depends strongly on the control and 
data flow structure of the program, with respect to the corresponding fault 
location, while the former depends on the type of statement at the fault 
location. 

The results of subsections 3.6 to 3.8 show that about 2/3 of the faults, 
on the average, appear in the functional modules of Attrition, Communica- 
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tion, and Observation which include approximately 50% of the program 
lines. This implies some mild tendency of the failure regions for functional 
clustering, since some functional modules are indeed more fault-prone than 
others. 

From subsection 3.9 we can see, comparing our results with Ginn's 
(1991), that for all pairs of structurally correlated failure regions, the 
clustering metric is, on the average, higher for functionally correlated ones. 
This is justified by fact that, in a reasonably well structured program, faults 
on the same group of functional modules will share a common control and 
data flow path segment more often than faults on different groups, therefore 
functional and structural correlation cannot be independent. 
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IV. CONCLUSIONS AND SUGGESTIONS FOR FURTHER RE- 



SEARCH 

It has been conjectured in the past that fault occurrences tend to 
converge on program locations, which implies that the revealing of a fault 
might indicate the existence of others in close proximity of location. 
However, the evidence so far has been mostly anecdotal. This thesis, together 
with Ginn's (1991) have been of the first to analyze the relationships between 
specific faults using structural (Ginn 1991), taxonomical and functional (this 
thesis) criteria. The results of both support the hypothesis of fault clustering 
and suggest methods for the exploitation of them in software testing. This 
chapter summarizes these results, in conjunction with previous work, and 
points towards the research questions which are open to further investigation. 

A. CONCLUSIONS 

This thesis, being a sequel to previous work by Ginn (1990), offers 
strong evidence that failure regions tend to form clusters, not only when 
structural criteria are used, but taxonomical and functional as well. 
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The clustering criteria used in this thesis were imposed externally, since 
both the fault taxonomy and the functional classification of the faults have 
been independent from the experimental data structure. Therefore, the 
observed clustering tendency of failure regions can be characterized as global 
as opposed to the local clustering tendency explored b Ginn (1990). In local 
clustering the criteria are strongly dependent upon the structure of the data 
at hand (Jain and Dubes 1988). Therefore the Jaccard coefficients used by 
Ginn readily fall into this category. 

The taxonomical clustering behavior suggests that parts of the program 
prone to control logic, case selection and control state faults, must be the 
focus of the testing effort. This implies that these parts of a program must 
also be thoroughly and extensively documented in order to facilitate this 
focus of effort. The CONFLICT experiment (Shimeall 1991a, b) data 
analysis suggests that decision points and program control flow have a higher 
probability of fault occurrence than other locations of the code. A good 
testing or documentation method, such as decision tables etc., is expected to 
reduce substantially the rate of this type of faults. 

The usefulness of the functional clustering behavior is that known faults 
in a program imply that the probability of more, undiscovered, faults in the 
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functional module is higher than the in rest of the code. This provides expe- 
rimental evidence in support of the anecdotal conjecture that faults tend to 
attract other faults (Myers 1979). It also suggests that the distribution of 
faults during the first tests indicates the most fault-prone functional modules, 
which should be singled out for additional testing. 

More often than not the testing effort is exponential, or of higher 
complexity, in the length of code. The ability to point out the most fault- 
prone modules or constructs, under the experimentally-verified assumption 
of functional and taxonomical clustering, represents a substantial reduction 
to the required amount of testing. 

The nature of the cluster formation, and the correlation to Ginns 
structural clustering, for the two criteria used was markedly different. The 
taxonomical classification tended to demonstrate a clustering of type C faults 
(and failure regions) in numbers one order of magnitude higher than the 
expected when a uniform fault per line of code distribution was assumed. 

In order to compare the taxonomical clustering of failure regions to the 
structural clustering of the same regions, we calculated both the average 
Jaccard coefficient and the average non-zero Jaccard coefficient (using Ginn's 
results) for every type of possible taxonomical clustering and every version 
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of the program. The results in Tables 3.2 and 3.3, in Chapter III, suggest that 
the structural clustering of failure regions does not depend on taxonomical 
clustering. This is attributed to the fact that structural clustering depends 
strongly on the control and data flow structure of a program with respect to the 
corresponding fault locations while the taxonomical clustering depends mainly on 
the type of program statement at the said locations. 

The functional criterion revealed a tendency of faults to concentrate on 
certain functional groups of modules (In the CONFLICT case in the Communication, 
Observation and Attrition groups). A quite interesting result, in this analysis, 
has been the small scale clustering exhibited by the faults, which tended to occur 
in high numbers within certain procedures (93) while the majority of the examined 
procedures (446 total, in all eight versions of CONFLICT) were faultless (cf. 
Appendix D for a detailed functional fault type distribution). Similar behavior 
of faults has been reported by Myers (1979) but no explanation was cited. The 
small-scale clustering suggests that some procedures are more complicated, 
therefore more fault-prone, than others. In the eight versions of the CONFLICT 
experiment, the average procedure was 30+20 lines of code in length while the 
average length of procedures with at least one fault has been 42.5±33 lines and 
with two or more faults 55±45 lines. This result provides evidence in support of 
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the above-mentioned argument about procedure complexity and fault 
clustering correlation, despite the inaccuracies introduced by the fact that 
only faults with a well-defined location were considered. 

Unlike the taxonomical-to-structural clustering lack of correlation, in 
testing the functional-to-structural clustering correlation in the same as above 
way, it was found that the average non-zero Jaccard coefficient is systemati- 
cally higher for functionally correlated pairs than for uncorrelated. This mild 
correlation is explained by the fact that in a reasonably well-structured 
program, faults of the same group of functional modules will share a 
common control and data flow path, will be structurally correlated, more 
often than faults on different groups. 

The use of the number of lines of code (excluding comments but not 
variable declarations) in our analysis instead of the Halstead length is due 
to the ease of use of this metric as well as to unpublished calculations. 
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B. COMPARISON OF RESULTS TO PREVIOUS WORK 



The results of this thesis generally support the findings of previous 
researchers in the area of relationships between faults. This agreement 
suggests that the relationships may not be specific to the CONFLICT 
software experiment but have a more general validity in large software 
applications. 

Myers (1979) postulates that functional clustering exists but provides 
no further evidence or explanation of the basis for it. This thesis, together 
with the companion work of Ginn (1990) makes a step toward identifying 
the specific behavior of faults that result in failure region clustering. 

The eminent role of control logic, case selection and control state (type 
C) faults in program testing has been always emphasized (Beiser 1990, 
Myers 1979), therefore the importance of taxonomical clustering cannot be 
easily overlooked. 

Briliant et al (1990), analyzing the faults in a 27-version software 
experiment, conclude that the faults across independent versions not only are 
not independent but the interdependence is, in many cases, more pronounced 
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among logically related (which includes taxonomically and functionally 
related) faults. 

Further study of the fault relations on the same version or across 
versions is required for establishment of the interaction mechanisms. 

C. SUGGESTIONS FOR FURTHER RESEARCH 

While the results of this work are promising, the experimental 
population was small and narrowly focused. Additionally, the programs were 
written by students. Both the method and the results should be validated 
using a broad range of professionally-produced applications. 

One weakness of the method used in this thesis is that in testing the 
taxonomical clustering hypothesis, a small number of faults (about 10%) 
cannot be classified and therefore they are not included in the analysis. This 
is expected to introduce some minor inaccuracies in the results. 

Another weakness is the use of seven functional groups of modules in 
the functional clustering analysis, Movement, Observation, Attrition, 
Communication, Environment, Restoration and Others. This coarse functional 
decomposition, imposed by the necessity of a common base for comparison 
for all eight versions of CONFLICT, blurs the small scale clustering. In 
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addition, the number of lines per module is quite a weak metric for a 
procedure complexity and cannot be used effectively as an oracle to single 
out the most fault-prone modules. However, this simple metric indicates that 
it is possible to use some normal metric as a predictor of the complexity of 
a program unit. Further research is required to develop metrics that can serve 
as oracles for fault prone modules. 

Finally, the Jaccard coefficient, based on the number of shared 
dimensions bounding two failure regions, is not a very efficient measure of 
structural clustering. A better way to establish a metric of two-fault 
correlation might be the use of the number of paths through both their 
respective locations. This method was not used in this thesis since we have 
been focused on taxonomical and functional rather than structural clustering. 
However, once certain ambiguities, such as distributed faults, or faults 
without a specific location, such as a missing function etc., are resolved the 
proposed metric will directly translate the relationship between two failure 
regions into two sets of code locations. 
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APPENDIX A: NOTATION AND SYMBOLS 



• Dj. Partition of the input domain, by path j 

• P(j): Boolean expression equal to the conjunction of all branch 
predicate constraints along path j. Therefore all elements of Dj satisfy 

PCD. 

• Bound(G): The boolean expression which defines failure region G. 

• M Symbol for exclusive or of boolean expressions. 

• S jI Sj 2 ..S jn (D r ): The mapping of a subset, D', of the input domain to the 
output range, by a sequence, S jl S j2 ..S jn , of program statements (this 
sequence is called the path product ), where j is a path. Condition 
evaluations are retained in the path products for consistency. In fact, the 
path products of the two segments starting from a decision point (such 
as an if C then else construct or a loop exit condition etc) are preceded 
with the condition C and condition -iC respectively, depending upon 
which part of the decision (if or else) results in their execution. 

• SjjSj/ SjjPv+i. . S jJ+m )*:S j>n : A sequence of program statements, when a 
subset of them, S j4 S j4+1 ..S j4+m , is the body of a loop. 

• SjjS ja ( SjjSjj+j' •Sj 2+m • A sequence of program statements, when 

a subset of them, S j4 S j4+1 ..S j4+m , is the body of a loop, which executes at 
least once. 

• SiG^Gj): The simple matching coefficient for failure regions Gj and Gj. 
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S(G„G,) = >^ 
a 

where a*,, stands for the non-bounding dimensions of both failure 
regions a n for the composite dimensions of Gj and Gj and a for the 
number of all input dimensions. 



♦ J(Gj,Gj ): The Jaccard coefficient for failure regions Gi and Gj. 



J(G p Gj) = 




a-a. 



oo 



where a^ stands for the non-bounding dimensions of both failure 
regions a n for the composite dimensions of Gj and Gj and a for all 
input dimensions. 



• A— »B or, equivalently, B— »A : Logical implication, A implies B ( or 
iAvB). 



• Identically participating dimensions : The dimensions that appear in 
both boundaries of a pair of failure regions in exactly the same way. 

• Coincidentally Participating Dimensions : The dimensions that appear 
in both boundaries of a pair of failure regions but not in an identical 
way. 

• Nonbounding Dimensions : The dimensions that, for a pair of failure 
regions, do not appear in the boundaries of both regions. The boun- 
daries these dimensions place on these failure regions are no more 
restrictive than their entire range of values. 

• Composite Dimensions : Collective name for both Identical and 
Coincidental dimensions 
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• Chi-Square Statistic (% 2 ): Statistic used to test whether a significant 
difference exists between an observed and an expected number of 
objects 8 (the expected number of objects results from an assumed 
distribution of objects into categories). The greater the chi-square 
statistic the lower the confidence that the sample data follow the 
assumed distribution. If there are k categories of objects, Oj is the 
number of observed and Ej the number of expected objects in category 
j, then the statistic has k-1 degrees of freedom and is equal to: 



j‘ i L j 



8 Siegel and Castellan 1988 
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APPENDIX B: TAXONOMICAL FAULT TYPE DISTRIBUTION 
In this Appendix we present histograms (Figures B1 to B8, in light 
grey) of the relative frequency of occurrence of fault types in the eight 
versions of CONFLICT (Shimeall 1991). The fault types in the histograms 
are LOOP, CONTROL, PROCESS and EXCEPTION HANDLING. They 
correspond to the faults of type A, B, C and D of Chapter Id (In fact A, B, 
C, D are just abbreviations). 

The fault type frequency histograms are compared with the expected 
frequency histograms of a simple model which assumes uniform distribution 
of faults (histograms in dark grey). In figure B9 we present an overall 
comparison of the observed to expected number of faults ratio, for all eight 
versions. There is a very sharp peak of the ratio distributions in the 
CONTROL type of faults, which implies that this type of fault has a 
frequency of occurrence much greater than the expected by the assumed 
model. 
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LOOP CONTROL PROCE8. EXCEPTION 



Figure Bl: Taxonomical Fault Frequency in Version I (light grey) versus 
random distribution of faults to program locations (dark grey). 
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LOOP CONTROL PROCES. EXCEPTION 



Figure B.2: Taxonomical Fault Frequency in Version II (light grey) 
versus random distribution of faults to program locations (dark grey). 
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LOOP CONTROL PROCES. EXCEPTION 



Figure B.3: Taxonomical Fault Frequency in Version III (light grey) 
versus random distribution of faults to program locations (dark grey). 



57 




LOOP CONTROL PROCES. EXCEPTION 



Figure B.4: Taxonomical Fault Frequency in Version IV (light grey) 
versus random distribution of faults to program locations (dark grey). 




Figure B.5: Taxonomical Fault Frequency in Version V (light grey) 
versus random distribution of faults to program locations (dark grey). 
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20 



16 



10 




LOOP CONTROL 




PROCES. EXCEPTION 



Figure B.6: Taxonomical Fault Frequency in Version VI (light grey) 
versus random distribution of faults to program locations (dark grey). 




Figure B.7: Taxonomical Fault Frequency in Version VII (light grey) 
versus random distribution of faults to program locations (dark grey). 
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LOOP CONTROL PROCES. EXCEPTION 



Figure B.8: Taxonomical Fault Frequency in Version VIII (light grey) 
versus random distribution of faults to program locations (dark grey). 




Figure B.9: Actual number of taxonomical faults to expected number of 
faults ratio for the eight versions of the program. 
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APPENDIX C: SHARED BOUNDING DIMENSIONS IN TAXONOMI- 



CALLY CLUSTERED FAILURE REGIONS 

In this Appendix we present a detailed analysis, whether taxonomical 
clustering of failure regions results in an increased number of shared 9 
bounding dimensions. 

The results, are presented in the tables Cl to C8. Each of the tables, 
corresponds to one of the eight versions of the CONFLICT program 
(Shimeall 1991, 1991b). Each table entry, corresponds to a pair of failure 
regions of the same version. It is noted that the numbering of the failure 
regions is not significant in this analysis. It merely represents the order in 
which the faults and the corresponding failure regions were discovered. 

Table entries on the main diagonal, contain the logical cluster identifier 
(A to D), for the corresponding failure region (cf. Chapter III). Each of the 
off diagonal entries contains the Jaccard coefficient for the identical 



In this analysis, shared dimensions are the ones that correspond to the same predi- 
cates, identical. 
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dimensions of the two failure regions labeling the row and column, and the 
fault type identifier, in case the regions belong to the same logical cluster. 

For example, entry (1.2, 1.5) contains 0.100./B which means that the 
Jaccard Coefficient for these failure regions is equal to 0.100., and that both 
correspond to faults of type B. On the other hand, entry (1.12,1.5) contains 
0.154, so the two failure regions have a Jaccard coefficient equal to 0.154, 
but correspond to different types of faults (D and B respectively, in this 
case). 
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TABLE C.l : 

JACCARD COEFFICIENTS OF FAILURE REGIONS OF VERSION I 





1.1 


1.2 


1.3 


1.6 


1.5 


1.6 


1.7 


0/B 


B 


- 


- 


- 


- 


1.3 


0 


0 


C 


- 


- 


- 


1.3 


0 


0 


1.333/C 


c 


- 


- 


1.5 


0/B 


0.100/B 


L 


0 


B 


- 


1.6 


0 


0 


0/C 


0/C 


0 


C 


1.7 


0 


0 


0/C 


0/C 


0 


0/C 


1.3 


0 


0.083/B 


0 


0 


0/B 


0 


1.9 


0/B 


0/B 


D 


0 


0/B 


0 


1.10 


0.333 


L 


0/C 


0/C 


0 


0/C 


1.11 


0.333 


0 


0/C 


0/C 


0 


0/C 


1.10 


0 


0.0833 


0 


0 


0.154 


0 


I 1.13 


0 


0 


0/C 


0/C 


0 


0/C 


1.14 


0/B 


0/B 


0 


0 


0/B 


0 
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TABLE C.l : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 



VERSION I 





1.7 


1.9 


1.9 


1.10 


1.11 


1.12 


1.13 


1.7 


C 


- 


- 


- 


- 


- 


- 


1.8 


0 


0 


- 


- 


- 


- 


- 


1.9 


0 


0.063/B 


0 


- 


- 


- 


- 


1.10 


0/C 


0 


0 


- 


- 


- 


- 


1.10 


0/C 


0 


0 


0.580/C 


C 


- 


- 


1.12 


0 


0 


0 


0 


0 


0 


- 


1.13 


0/C 


0.182 


0.083 


0/C 


0/C 


0 


c 


1,4 


0 


0.067/B 


0 


0 


0 

1 


0 

1 


0.143 

J 



TABLE C.l : JACCARD COEFFICIENTS OF FAILURE REGIONS 



OF VERSION I 





1.1 


1.2 


1.3 


1.1 


1.5 


1.1 


1.7 


1.17 


0.333NB 


(KB 


0 


0 


CKB 


0 


0 


1.18 


0 


0.100 


0 


0 


0.053 


0 


0 


1.18 


0 


0 


C KC 


(KC 


0.250 


(KC 


( K c 


1.20 


CKB 


0.056VB 


0 


0 


0.053XB 


0 


0 


1.22 


(KB 


CKB 


0 


0 


(KB 


0 


0 


1.18 


0 


0 


(KC 


OvC 


0 


(KC 


(KC 


1.28 


0 


0 


(KC 


(KC 


0.053 


(KC 


(KC 


1.27 


0 


0 


(KC 


OvC 


0 


(KC 


(KC 


1.28 


0 


0 


0\C 


0\C 


0.500 


(KC 


(KC 
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TABLE C.l : JACCARD COEFFICIENTS OF FAILURE REGIONS 

OF VERSION I 





1.8 


1.9 


1.10 


1.11 


1.12 


1.13 


1.14 


1.27 


0\B 


CKB 


0.500 


0.500 


0 


0 


(KB 


1 118 


0 


0 


0 


0 


0.25CKD 


0 


0.111 


1.19 


0 


0.154 


CKC 


CKC 


0.100 


0.125\C 


0.067 


1.18 


0.118KB 


0.048KB 


0 


0 


0.053 


0.133 


0.105\B 


1.1S 


0.056KB 


0.60CKB 


0 


0 


0 


0.071 


0.048\B 


1.23 


0 


0 


(KC 


(KC 


0.700 


(KC 


0 


1.1S 


0.050 


0.045 


(KC 


(KC 


0 


0.118\C 


0.109 


1.27 


0.120 


0.036 


(KC 


(KC 


0 


0. 143\C 


0.074 


1.28 


0 


0 


0\C 


(KC 


0.143 


(KC 


0 
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TABLE C.l : JACCARD COEFFICIENTS OF FAILURE REGIONS 

OF VERSION I 





1.17 


i.i? 


1.17 


1.20 


1.1S 


e 


- 


- 


- 


1.18 


0 


0 


- 


- 


1.18 


0 


0.100 


- 


- 


1.2? 


0/B 


0.100 


0.118 


B 


1.18 


o/b 


0 


0.067 


0.043\B 


1.23 


0 


0.250 


0.09 1\C 


0.050 


1.18 


0 


0 


0.059\C 


0.421 


1.27 


0 


0 


0.045XC 


0.103 


1.28 


0 


0.071 


0.200KC 


0.050 



TABLE C.l : JACCARD COEFFICIENTS OF FAILURE REGIONS 

OF VERSION I 





1.22 


1.23 


1.25 


1.27 


'• 2 * | 


1.22 


B 


- 


- 


- 




1.23 


e 


C 


- 


- 


- 


1.25 


0.042 


0.048\C 


C 


- 


- 


1.27 


0.033 


0\C 


0.06 1\C 


C 


- 


1.28 


0 


0.357\C 


0.05 3\C 


CKC 


c 
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TABLE C.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.1 


2.2 


2.3 


2.4 


2.5 


2.6 


2.1 


B 


- 




- 


- 


- 


2.2 


0/B 


B 


- 


- 


- 


- 


2.1 


0/B 


0.09 1/B 


B 


- 


- 


- 


2.4 


0 


0.091 


0.300 


C 


- 


- 


2.5 


0 


0.067 


0.133 


0.133/C 


C 


- 


2.1 


0 


0.091 


0.091 


0.083 


0.063 


D 


2.1 


0.077/B 


0/B 


0/B 


0 


0 


0 


2.1 


0.077/B 


0/B 


0/B 


0 


L 


0 


2.11 


0/B 


0/B 


0/B 


0 


D 


0 


2.11 


0/B 


0/B 


0/B 


0 


0 


0 


2.12 


0/B 


0/B 


0/B 


0 


L 


0 


2.13 


0/B 


0.111/B 


0.09 1/B 


0.091 


0.154 


0 


2.14 


0/B 


0/B 


0/B 


0 


0 


0 


2.15 


0/B 


0/B 


0/B 


0 


0 


0 
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TABLE C.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.7 


2.8 


2.10 


2.11 


2.12 


2.13 


2.14 


2.15 


2.7 


B 


- 


- 


- 


- 


- 


- 


- 


2.8 


0.636/B 


B 


- 


- 


- 


- 


- 


- 


2.10 


o/b 


0/B 


B 


- 


- 


- 


- 


- 


2.11 


o/b 


0/B 


0/B 


B 


- 


- 


- 


- 


2.12 


0/B 


0/B 


0/B 


0/B 


B 


- 


- 


- 


2.13 


0/B 


0/B 


0/B 


0/B 


0/B 


B 


- 


- 


2.14 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


B 


- 


2.15 


0/B 


0/B 


0/B 


0/B 


0/B 


0.400/B 


0/B 


B 
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TABLE C.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.1 


2.2 


2.3 


2.4 


2.5 


2.6 


2.16 


CKB 


(KB 


(KB 


0 


0 


0 


2.17 


CKB 


(KB 


(KB 


0 


0 


0 I 


2.18 


(KB 


(KB 


CKB 


0 


0 


0 


i 2.19 


CKB 


(KB 


CKB 


0 


0 


0 


2.20 


(KB 


CKB 


(KB 


0 


0 


0 


2.21 


(KB 


(KB 


(KB 


0 


0 


0 


2.21 


CKB 


CKB 


(KB 


0 


0 


0 


2.23 


(KB 


(KB 


(KB 


0 


0 


0 


2.25 


(KB 


(KB 


CKB 


0 


0 


0 


2.30 


(KB 


O.lllNB 


O.lllNB 


0.100 


0.071 


0.333 
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TABLE C.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.7 


2.8 


2.10 


2.11 


2.12 


2.13 


2.14 


2.15 


2.18 


(KB 


(KB 


(KB 


(KB 


CKB 


(KB 


0.400/B 


0.40CKB 


2.22 


CKB 


(KB 


CKB 


(KB 


CKB 


CKB 


0.400/B 


0.40CKB 


2.18 


CKB 


CKB 


CKB 


(KB 


CKB 


CKB 


0.400/B 


0.40CKB 


2.18 


CKB 


(KB 


(KB 


(KB 


(KB 


CKB 


(KB 


(KB 


2.18 


CKB 


(KB 


CKB 


(KB 


CKB 


CKB 


(KB 


CKB ' 


2.21 


CKB 


(KB 


CKB 


(KB 


CKB 


CKB 


CKB 


CKB 


2.22 


(KB 


CKB 


(KB 


(KB 


CKB 


CKB 


CKB 


CKB 


2.21 


(KB 


(KB 


(KB 


(KB 


CKB 


CKB 


(KB 


CKB 


2.25 


(KB 


CKB 


(KB 


(KB 


(KB 


(KB 


(KB 


(KB ; 


2.30 


(KB 


(KB 


CKB 


CKB 


(KB 


(KB 


CKB 


CKB 
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TABLE C.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 



r— 


2.19 


2.11 


2.18 


2.19 


2.20 


2.16 


B 


- 


- 


- 


- 


2.17 


0.400/B 


B 


- 


- 


- 


2.18 


m 


0.400/B 


B 


- 


- 


2.18 


0/B 


0/B 


0/B 


B 


- 


2.20 


0/B 


0/B 


0/B 


0.57 1/B 


B 


2.21 


0/B 


0/B 


0/B 


0.571/B 


0.571/B 


2.20 


0/B 


0/B 


0/B 


0.571/B 


0.571/B 


2.23 


0/B 


0/B 


0/B 


0.571/B 


0.571/B 


2.25 


0/B 


0/B 


0/B 


0/B 


0/B 


2.30 


0/B 


0/B 


0/B 


0/B 


0/B 



TABLE C.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.21 


2.22 


2.23 


2.24 


2.30 


2.22 


B 


- 


- 


- 


- 


2.22 


0.571/B 


B 


- 


- 


- 


2.23 


0.571/B 


0.571/B 


B 


- 


- 


2.25 


0/B 


0/B 


0/B 


B 


- 


2.30 


0/B 


0/B 


0/B 


0/B 


B 
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TABLE C.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS 

OF VERSION m 





3.1 


3.2 


3.3 


3.4 


3.5 


3.6 


3.7 


3.3 


3.10 


II 31 


e 


- 


- 


- 


- 


- 


- 


- 


- 


3.2 


0.600- 

/B 


B 


- 


- 


- 


- 


- 


- 




3.3 


0.250- 
/ B 


0.286- 

/B 


B 


- 


- 


- 


- 


- 


- 


3.4 


0/B 


0/B 


0/B 


0 


- 


- 


- 


- 


- 


3.5 


0/B 


0/B 


0/B 


0/B 


B 


- 


- 


- 


- 


3.6 


0.143 


0.200 


0 


0 


0 


A 


- 


- 


- 


3.7 


0.087 


0.071 


0 


0 


0 


0.071 


- 


- 


- 


3.9 


0 


0 


0 


0 


0 


0 


0.476 


C 


- 


3.16 


0 


0 


0 


0 


0.167 


0 


0 


0/C 


C 


3.11 


0.111/B 


0.125/B 


0.091/B 


0/B 


0/B 


0.125 


0.059 


0 


0 


3.21 


0.111/B 


0.125/B 


0/B 


0/B 


0/B 


0.11 


0.188 


0.087 


0 


3.13 


0 


0 


0 


0 


0 


0 


0 


0.042/C 


0/C 


3.15 


0.077 


0.083 


0.364 


0.100 


0 


0 


0 


0.040/C 


0/C 


3.16 


0.11 


0.125 


0.091 


0 


0 


0.125 


0.059 


0 


0 


3.21 


(KB 


0/B 


0/B 


0/B 


0/B 


0 


0 


0 


0 


3.16 


0.087 


0.077 


0 


0 


0 


0.091 


0.714 


0.429/C 


0/C 


3.21 


0 


0 


0 


0 


0 


0 


0 


0 


0 


3.22 


0.125- 

/B 


0.167- 

/B 


0/B 


0/B 


0/B 


0.200 


0.063 


0.048 


0 


3.23 


0.182/B 


0.200/B 


0.077/B 


0/B 


0.0833/B 


0.091 


0.111 


0.040 


0.111 
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TABLE C.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 





3.11 


3.11 


3.13 


3.14 


3.16 


3.17 


3.16 


3.21 


3.22 


3.2 
3 1 


3.11 


B 


- 


- 


- 


- 


- 


- 


- 




- 


3.12 


0.111- 

/B 


B 


- 


- 


- 


- 


- 


- 


- 


• 


3.13 


0 


0 


C 


- 




- 


- 


- 


- 


- 


3.15 


0.067 


0 


0/C 


c 


- 


- 


- 


- 


- 


- 


3.18 


0.333 


0.111 


0 


0.067 


0 


- 


- 


- 


- 


- 


3.17 


0/B 


0/B 


0 


0 


0 


9 


- 


- 


- 


- 


3.18 


0.063 


0.059 


0/C 


0.056 


0.063 


0 


C 


- 


- 


- 


3.21 


0 


0 


0 


0 


0/D 


0 


0 


D 


- 


- 


3.22 


0.111- 
/ B 


0.100- 

/B 


c 


0 


0.111 


0/B 


0.154 


0 


B 


- 


3.23 


0.077- 

/B 


0.167- 

/B 


0 


0.063 


0 


0 


0.053 


0 


0.833- 

/B 


B 

- 
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TABLE C.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 





3.1 


3.2 


3.3 


3.4 


3.5 


3.6 


3.2 


3.9 


3.10 


3.24 


0.143 


0.167 


0.167 


0 


0.286 


0 


0.071 


0/C 


0.250- 

/c 


3.25 


0.125 


0.167 


0.250 


0 


0 


0.250 


0.067 


0/C 


0/C 


3.26 


0/B 


0/B 


0.125/B 


0.125/B 


0.571/B 


0 


0 


0 


0.167 


3.2S 


0/B 


0.125/B 


0/B 


0.100/B 


0.222/B 


0.125 


0.063 


0 


0 


3.34 


0/B 


0/B 


0.167/B 


0/B 


0/B 


0 


0 


0 


0 


3.34 


0/B 


0/B 


0.143/B 


0/B 


0/B 


0 


0 


0 


0 


3.34 


0/B 


0/B 


0.143/B 


0/B 


0/B 


0 


0 


0 


0 


3.35 


0/B 


0/B 


0.143/B 


0/B 


0/B 


0 


0 


0 


0 


3.34 


0/B 


0/B 


0.167/B 


0/B 


0/B 


0 


0 


0 


0 


3.37 


0/B 


0/B 


0.143/B 


0/B 


0/B 


0 


0 


0 


0 


3.35 


0.111/B 


0/B 


0/B 


0.125/B 


0.222/B 


0 


0.067 


0 


0.167 


3.39 


0/B 


0/B 


0/B 


0.125/B 


0.222/B 


0 


0 


0 


0.200 


3.40 


0/B 


0/B 


0/B 


0.125/B 


0.222/B 


0 


0 


0 


0.200 


3.43 


0.071/B 


0/B 


0/B 


0.125/B 


0.222/B 


0 


0 


0 


0.200 


3.4# 


0/B 


0.111/B 


0/B 


0.126/B 


0.222/B 


0 


0 


0 


0.200 


3.43 


0.091/B 


0/B 


0/B 


0.167/B 


0/B 


0.143 


0.118 


0.042 


0 


I 3.44 j 


0/B 


0.083/B 


0/B 


0/B 


0/B 


i 0.091 


0.770 


0.381 


0 


3.45 


0/B 


0/B 


0/B 


0/B 


0.056/B 


0 


0 


0.529 


0.381 
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TABLE C.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 





3.11 


3.11 


3.13 


3.15 


3.16 


3.17 


3.18 


3.21 


3.22 


3.23 


3.24 


0 


0 


0/C 


0.10- 

0/C 


0 


0 


0.07- 

7/C 


0 


0 


0.18 

2 


3.25 


0.111 


0.100 


0/C 


0.09- 

1/C 


0.11 

1 


0 


0.08- 

3/C 


0 


0.16 

7 


0.08 

3 


3.26 


0/B 


0/B 


0 


0.091 


0 


0/B 


0 


0 


0/B 


0.0- 
83- 
/ B 


3.28 


0.11- 

1/B 


0.11- 

1/B 


0 


0 


0.11 

1 


0/B 


0.063 


0 


0.11- 

1/B 


0.0- 
83- 
/ B 


3.32 


0/B 


0/B 


0 


0.100 


0 


0/B 


0 


0.33 

3 


0/B 


0/B 


3.33 


0/B 


0/B 


0 


0.091 


0 


0/B 


0 


0.25 

0 


0/B 


0/B 


3.34 


0/B 


0/B 


0 


0.091 


0 


0/B 


0 


0.25 

0 


0/B 


0/B 


3.35 


0/B 


0/B 


0 


0.091 


0 


0/B 


0 


0.25 

0 


0/B 


0/B 
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TABLE C.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 



1 


3.11 


3.12 


3.13 


3.15 


3.16 


3.17 


3.18 


3.21 


3.22 


3.23 


1 3,39 


0/B 


0/B 


0 


0.09 

1 


0 


0/B 


0 


0.25 

0 


0/B 


0/B 


3.37 


0/B 


0/B 


0 


0.09 

1 


0 


0/B 


0 


0.25 

0 


0/B 


0/B 


3.3? 


0/B 


0/B 


0 


0 


0 


0/B 


0.063 


0 


0/B 


0.08- 

3/B 


3.39 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0/B 


0.08- 

3/B 


3.44 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0/B 


0.08- 

3/B 


3.42 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0/B 


0.08- 

3/B 


3.42 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0/B 


0.08- 

3/B 


3.43 


0.08- 

3/B 


0.07- 

7/B 


0 


0 


0.08 

3 


0/B 


0.143 


0 


0 . 111 - 

/B 


0.06- 

7/B 


3.44 


0.06- 

7/B 


0.06- 

3/B 


0 


0 


0.06 

7 


0/B 


0.910 


0 


0.077- 

/B 


0.05- 

6/B 


3.45 


0/B 


0.05- 

0/B 


0 


0 


0 


0/B 


0.471 


0 


0/B 


0/B ' 
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TABLE C.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION III 





3.24 


3.25 


3.26 


3.28 


3.32 


3.33 


3.34 


3.35 


3.24 


C 


- 


- 


- 


- 


- 


- 


- 


; 3.2$ 


0 


C 


- 


- 


- 


- 


- 


- 


3.26 


0.333 


0 


B 


- 


- 


- 


- 


- 


3.28 


0.125 


0.111 


0.333- 

/B 


B 


- 


- 


- 


- 


3.3* 


0 


0 


0.167/B 


0/B 


B 


- 


- 


- 


3.33 


0 


0 


0.143/B 


0/B 


0.500/B 


B 


- 


- 


3.38 


0 


0 


0.143/B 


0/B 


0.500/B 


0.400/B 


B 


- 


3.3* 


0 


0 


0.143/B 


0/B 


0.500/B 


0.400/B 


0.400/B 


B 


3.38 


0 


0 


0.143/B 


0/B 


0.500/B 


0.400/B 


0.400/B 


0.400/B 


3.37 


0 


0 


0.143/B 


0/B 


0.500/B 


0.400/B 


0.400/B 


0.400/B 


3.38 


0.286 


0 


0.375/B 


0.200/B 


0/B 


0/B 


0/B 


0/B 


3.38 


0.143 


0 


0.250/B 


0.400/B 


0/B 


0/B 


0/B 


0/B 


3.40 


0.143 


0 


0.250/B 


0.400/B 


0/B 


0/B 


0/B 


0/B 


3.42 


0.143 


0 


0.250/B 


0.100/B 


0/B 


0/B 


0/B 


0/B 


3.42 


0.143 


0 


0.250/B 


0.100/B 


0/B 


0/B 


0/B 


0/B 


3.43 


0 


0.125 


0/B 


0.083/B 


0/B 


0/B 


0/B 


0/B 


3.44 


0.077 


0.083 


o/b 


0.063/B 


0/B 


0/B 


0/B 


0/B 


3.45 


0.059 


0 


0.059/B 


0.053/B 


0/B 


0/B 


0/B 


0/B 
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TABLE C.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION HI 





3.36 


3.37 


3.38 


3.39 


3.44 


3.41 


3.42 


3.43 


3.44 


3.45 


3.36 


B 


- 


- 


- 


- 


- 


- 


- 


- 


- 


3.37 


0.400- 

/B 


B 


- 


- 


- 


- 


- 


- 


- 


- 


3.38 


0/B 


0/B 


B 


- 


- 


- 


- 


- 


- 


- 


3.39 


0/B 


0/B 


0.286/B 


B 


- 


- 


- 


- 


- 


- 


3.40 


0/B 


0/B 


0.286/B 


0.500/B 


B 


- 


- 


- 


- 


- 


3.41 


0/B 


0/B 


0.286/B 


0.500/B 


0.500- 
/ B 


B 


- 


- 


- 


- 


3.41 


0/B 


0/B 


0.286/B 


0.500/B 


0.500- 
/ B 


0.500/ 

B 


B 


- 




- 


3.43 


0/B 


0/B 


0/B 


0.091/B 


0.091/ 

B 


0.091/ 

B 


0.091/ 

B 


B 


- 


- 


3.44 


0/B 


0/B 


0.067/B 


0/B 


0/B 


0/B 


0/B 1 


0.143 
/ B 


B 


• 


3.45 


0/B 


0/B 


0.105/B 


0.053/B 


0.053/ 

B 


0.053/ 

B 


0.053/ 

B 


0.053 

/B 


0/B 


B 
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TABLE C.4 : JACCARD COEFFICIENTS OF FAILURE REGIONS 



OF VERSION IV 





4.1 


4.2 


4.3 


4.9 


4.5 


4.7 


4.8 


4.9 


4.10 


4.11 


4.12 1 


4.1 


C 


- 


- 


- 


- 


- 


C 


- 


- 


- 


- | 


4.2 


0 


e 


- 


- 


- 


- 


- 


- 


- 


- 


- 




4.1 


0/C 


0.118 


- 


- 


- 


- 


c 


- 


- 


- 




4.4 


0/C 


0 


0.045/ 

C 


c 


- 


• 


- 


- 


- 


- 


- 


4.5 


0 


0/B 


0.063 


0.071 


e 


- 


- 


- 


- 


- 


- 


4.4 


0 


0/B 


0.056 


0.071 


0.167/ 

B 


B 


- 


- 


- 


- 


- 


4.8 


0 


0/B 


0.167 


0.059 


0.125/ 

B 


0.111/ 

B 


B 


- 


- 


- 


• 


4.4 


0/C 


0 


0.063/ 

C 


0.071/ 

C 


0.250 


0.167 


0.125 


C 


- 


- 


- 


4.1 6 


0 


0/B 


0.056 


0.063 


0/B 


0.125/ 

B 


0.091/ 

B 


0.125 


V 


- 


- 


4.14 


0/C 


0.053 


0.043/ 

C 


0/C 


0 


0 


« 


0/C 




C 


- 


4.12 


0/C 


0 


0.063/ 

C 


0.214/ 

C 


0.167 


0.125 


0.100 


0.167/ 

C 


0.100 


0/C 


c 


4.24 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


4.24 


0/C 


0 


0/C 


0/C 


0 


0 


€ 


0/C 


0 


0/C 


0/C 


4.14 


0.500 


0/B 


0.091 


0 


0/B 


0/B 


0.214/ 

B 


0 


0/B 


0 


0 


4.16 


! 


0.071/ 

B 


0 


0.188 


0/B 


0/B 


0/B 


0 


0/B 


0 


0.077 


4.24 


C/C 


0.100 


0/C 


0/C 


0 


0 


0 


0/C 


0 


0/C 


0/C 


4.19 


0 


0.100 


0 


0 


0 


0 


0 


0 


0 


0 


0 


4.21 


0 


0.071 


0 


0 


0 


0 


0 


0 


0 


0 


0 


4.24 


0 


0/B 


0 


0 


0/B 


0/B 


0/B 


0 


0/B 


0 


0 


4.24 


0/C 


0.100 


0/C 


0 


0 


0 


0 


0/C 


0 


0/C 


0/C 


1-2^ 


0/C 


0.083 


0/C 


0 


0 

i i 


0 

1 1 


0 


0/C 


0 


0/C 


0/C I 
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TABLE C.4 : JACCARD COEFFICIENTS OF FAILURE REGIONS 

OF VERSION IV 





4.13 


4.14 


4.15 


4.16 


4.18 


4.19 


4.21 


4.22 


4.24 


4.2 

8 


1 4.13 


A 


- 


- 


- 


- 


- 


- 


- 


- 


- 


4.14 


0.800 


c 


- 


- 


- 


- 


- 


- 


- 


- 


4.15 


0 


0 


9 


- 


- 


- 


- 


- 


- 


- 


4.15 


0 


0 


0/B 


0 


- 


- 


- 


- 


- 


- 


4.18 


0 


0/C 


0 


0.091 


C 


- 


- 


- 


- 


- 


4.18 


0 


0 


0 


0.071 


0.125 


D 


- 


- 


- 


- 


4.21 


0 


0 


0 


0.063 


0.083 


0.077/ 

D 


D 




- 


- 


4.22 


0 


0 


0/B 


0 


0 


0 


0 


e 


- 


- 


4.24 


0 


0/C 


0 


0.083 


0.167/C 


0.143 


0 


0 


c 


- 


4.21 


0 


0/C 


0 


0.071 


0.125/C 


0.111 


0.077 


0 


0.60 

/c 


C 
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TABLE C5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 



VERSION V 





5.1 


5.2 


5.3 


5.1 


5.5 


5.6 


5.7 


5.8 


5.9 


5.3 


B 


- 


- 


- 


- 


- 


- 


- 


- 


5.8 


(KB 


B 


- 


- 


- 


- 


- 


- 


- 


5.3 


(KB 


(KB 


0 


- 


- 


- 


- 


- 


- 


5.3 


(KB 


0.053VB 


CKB 


0 


- 


- 


- 


- 


- 


5.1 


0 


0.059 


0 


0.375 


C 


- 


- 


- 


- 


5.8 


(KB 


CKB 


0.167\B 


(KB 


0 


B 


- 


- 


- 


5.8 


(KB 


(KB 


(KB 


CKB 


0 


CKB 


B 


- 


- 


5.8 


0 


0.111 


0 


0.059 


0.067VC 


B 


13 


C 


- 


5.9 


(KB 


0.053\B 


CKB 


CKB 


0 


CKB 


(KB 


0 


B 


5.18 


CKB 


(KB 


CKB 


(KB 


0 


CKB 


(KB 


13 


0.143\B 


5.11 


(KB 


(KB 


(KB 


CKB 


0 


CKB 


(KB 


13 


(KB 


5.18 


(KB 


0\B 


(KB 


CKB 


0 


(KB 


0\B 


0 


OvB 


5.18 


0 


0.050 


D 


0.083 


0.100 


0 


I) 


0.056 


0 


5.18 


(KB 


(KB 


(KB 


(KB 


0 


(KB 


(KB 


0 


(KB 


5.16 


(KB 


(KB 


(KB 


CKB 


0 


(KB 


CKB 


13 


(KB 


5.11 


(KB 


CKB 


(KB 


CKB 


0 


CKB 


(KB 


13 


(KB 


5.18 


(KB 


(KB 


(KB 


CKB 


0 


CKB 


(KB 


0 


(KB 


II 519 


(KB 


(KB 


(KB 


CKB 


0 


CKB 


(KB 


0 
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TABLE C5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.10 


5.11 


5.12 


5.13 


5.15 


5.16 


5.1 

7 


5.18 


5.17 


| 5.10 


9 


- 


- 


- 


- 


- 


- 


- 


- 


5.11 


0\B 


B 


- 


- 


- 


- 


- 


- 


- 


5.12 


0\B 


0.20CN 

B 


B 


- 


- 




- 


- 


- 


5.13 


0 


B 


0 


0 


- 


- 


- 


- 


- 


5.10 


0\B 


(KB 


(KB 


0 


B 


- 


- 


- 


- 


5.16 


(KB 


(KB 


(KB 


0 


0.714 

\B 


B 


- 


- 


- 


5.11 


(KB 


(KB 


CKB 


0 


(KB 


(KB 


B 


- 


- 


5.18 


(KB 


(KB 


(KB 


0 


(KB 


(KB 


(KB 


B 


- 


5.19 


(KB 


(KB 


(KB 


0 


(KB 


(KB 


(KB 


(KB 


B 
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TABLE C5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.1 


5.2 


5.3 


5.6 


5.5 


5.6 


5.7 


5.8 


5.28 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.2S 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.22 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.23 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.28 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.25 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.28 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.27 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.28 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.30 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.31 


0 


0 


0 


0 


0/C 


0 


0 


0 


5.31 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.31 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.34 


0 


0 


0 


0 


0/C 


0 


0 


0 


5.35 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


5.37 


0 


0 


0 


0 


0 


0 


0 


0 


5.30 


0 


0 


0 


0 


0 


0 


0 


0 


|j 5.40 


0 


0 


0 


0 


0 


0 


0 


— 2 — 1 



83 



TABLE C5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.9 


5.10 


5.11 


5.12 


5.13 


5.15 


5.16 


5.1 

7 


5.18 


5.19 


5.24 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.21 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.2S 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.23 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.24 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.25 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.24 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.27 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.28 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.3? 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.37 


0 


0 


0 


0 


0 


0.130 


0.261 


0 


0 


0 


5.32 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.33 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


5.34 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.35 


0/B 


0/B 


0/B 


0/B 


0 


0.040/ 

B 


0.040/ 

B 


0/B 


0/B 


0/B 


5.37 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.37 


0 


0 


0 


0 


0 


0.083 


0.083 


0 


0 


0 


5.40 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 
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TABLE C5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.19 


5.20 


5.21 


5.22 


5.23 


5.24 


5.25 


5.26 


5.27 


5.20 


0/B 


B 


- 


- 


- 


- 


- 


- 


- 


5.26 


0.444/ 

B 


0/B 


B 


- 


- 


- 


- 


- 


- 


5.22 


0/B 


0.286/B 


0/B 


B 


- 


- 


- 


- 


- 


5.23 


0.444/ 

B 


0/B 


0.444/ 

B 


0/B 


B 


- 


- 


- 


- 


5.26 


0/B 


0.286/B 


0/B 


0.286/ 

B 


0/B 


B 


- 


- 


- 


5.25 


0.444/ 

B 


0/B 


0.444/ 

B 


0/B 


0.444/ 

B 


0/B 


B 


- 


- 


5.26 


0/B 


0.286/B 


0/B 


0.286/ 

B 


0/B 


0.286/ 

B 


0/B 


B 


- 


5.27 


0.444/ 

B 


0/B 


0.444/ 

B 


0/B 


0.444/ 

B 


0/B 


0.444 

/B 


0/B 


B 


5.20 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


5.30 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


5.31 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.31 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


5.33 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


5.31 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.30 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


5.37 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.39 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.40 


0 


0 


0 


0 


0 


0 


0 


0 


0 
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TABLE C5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.28 


5.33 


5.31 


5.32 


5.33 


5.34 


5.35 


5.37 


5.39 


5.28 


B 


- 


- 


- 


- 


- 


- 


- 


- 


5.30 


0/B 


B 


- 


- 


- 


- 


- 


- 


- 


5.33 


0 


0 


C 


- 


- 


- 


- 


- 


- 


5.32 


0/B 


0/B 


0/B 


B 


- 


- 


- 


- 


- 


5.33 


0/B 


0/B 


0/B 


0/B 


B 


- 


- 


- 


- 


5.33 


0 


0 


0/C 


B 


0 


C 


- 


- 


- 


5.35 


0/B 


0/B 


0.053/ 

B 


0/B 


0/B 


0.214/ 

B 


B 


- 


- 


5.37 


0 


0 


0 


0 


0 


0 


0 


- 


1 


5.39 


0 


0 


0.125 


0 


0 


0 


0.063 


0 


- 


5.40 


0 


0 


0 


0 


0 


0 


0 


0 


0.053 
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TABLE C.6 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 



VERSION VI 



\ 


6.2 


6.2 


6.3 


6.6 


6.6 


6.6 


6.2 


6.8 


6.1 


C 


- 


- 


- 


- 


- 


- 


- 


6.4 


0 


D 


- 


- 


- 


- 


- 


- 


6.3 


0.357/ 

C 


0 


- 


- 


- 


- 


- 


- 


6.4 


0 


0 


D 


• 


- 


- 


- 


- 


6.5 


0/C 


0 


O/C 


0 


C 


- 


- 


- 


6.4 


0.077 


0 


0.063 


0 


0 


D 


- 


- 


6.7 


0.056/ 

C 


0 


0.048/ 

C 


0 


o/c 


0.091 


- 




6.7 


0.167/ 

C 


0.143 


O/C 


0 


o/c 


0.091 


0.500/ 

C 


C 


6.9 


0.059/C 


0 


0.059/C 


0 


o/c 


0.100 


0.286/C 


0.286/C 


6.11 


o/c 


0.143 


o/c 


0 


o/c 


0 


o/c 


o/c 


6.12 


o/c 


0 


o/c 


0 


o/c 


0 


o/c 


o/c 


6.13 


0 


0/B 


D 


0 


0 


0 


0 


0 


6.14 


o/c 


0 


O/C 


0 


o/c 


0 


o/c 


o/c 


6.15 


o/c 


0.111 


o/c 


0 


o/c 


0 


o/c 


o/c 


6.16 


0.364/C 


0 


0.286/C 


0 


o/c 


0.125 


0.083/C 


0.250/C 


6.17 


0.056 


/B 


0.048 


0 


0 


0.083 


0.091 


0.091 


6.18 


0/C 


0 


O/C 


0 


o/c 


0 


O/C 


O/C 
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TABLE C.6 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION VI 





6.1 


6.2 


6.3 


6.6 


6.6 


6.6 


6.7 


6.8 


6.20 


0 


0/B 


0 


0 


0 


0 


0 


0 


6.23 


0.056 


0/B 


0.048 


0 


0 


0.091 


0.091 


0.091 


6.24 


0/C 


0 


0/C 


0 


0/C 


0 


0/C 


0/C 



TABLE C.6 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION VI 





6.9 


6.11 


6.12 


6.13 


6.14 


6.1 

5 


6.1 

6 


6.1 

7 


6.18 


6.20 


6.2 

3 


6.2 

4 


6.9 


C 






















- 


6.11 


0/C 


C 






















6.11 


0/C 


0/C 


c 


- 


















6.13 


0 


0 


0 


e 


















6.14 


0/C 


0/C 


0.059/ 

C 


0 


C 


- 


- 


- 


- 


- 


- 


- 


6.15 


0/C 


0.143/ 

C 


0.071/ 

c 


0 


o/c 


C 


- 




- 


- 


- 


- 


6.18 


0.07 

1/C 


o/c 


o/c 


0 


o/c 


o/c 


c 


- 


- 


- 


- 


- 


6.17 


0.06 

7 


0 


0 


0/B 


0 


0 


0.0 

77 


B 


- 


- 


- 


- 


6.18 


0/C 


o/c 


0.06 

7/C 


0 


0.40 

0/C 


o/c 


0/C 


0 


C 


- 


- 


- 


6.20 


0 


0 


0 


0/B 


0 


0 


0 


0/B 


0 


B 


- 


- 


6.23 


0.07 

1 


0 


0 


0/B 


0 


0 


0.0 

77 


0.73/ 

B 


0 


0/ 

B 


B 


- 


6.24 


0/C 


0.071/ 

C 


0.118/ 

C 


0 


0.27 

8/C 


0.063 

/ c 


0/C 


0 


0.46 

7/C 


0 


0 


C 
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TABLE C.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 



VERSION vn 



- 


7.1 


7.2 


7.3 


7.4 


7.5 


7.6 


7.7 


7.1 


B 


- 


- 


- 


- 


- 


- 


7.2 


0.083/B 


e 


- 


- 


- 


- 


; 


7.4 


0.083/B 


0.818/B 


e 


- 


- 


- 


- 


7.4 


0/B 


0/B 


0/B 


0 


- 


- 


- 


7.5 


0.167 


0 


0 


0.071 


- 


- 


- 


7.6 


0/B 


0/B 


0/B 


0/B 


0 


B 


- 


7.2 


0/B 


0/B 


0/B 


0/B 


0 


0.333/B 


0 


7.2 


0/B 


0/B 


0/B 


0/B 


0 


0.333/B 


0.500/B 


7.9 


0 


0 


0 


0.333 


0 


0 


0 


7.12 


0 


0.231 


0.214 


0 


0/C 


0 


0 


7.11 


0 


0 


0 


0 


0 


0.200 


0.167 


7.12 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


7.13 


0 


0 


0 


0 


0 


0 


0 


7.12 


0 


0 


0 


0.143 


0/C 


0 


0 


7.15 


0.200 


0.091 


0.083 


0 


0/C 


0 


0 


7.16 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


0/B 
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TABLE C.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.8 


7.9 


7.10 


7.11 


7.12 


7.13 


7.14 


7.15 


7.15 


7.8 


B 


- 


- 


- 


- 


- 


- 


- 


- 


7.8 


0 


- 


- 


- 


- 


- 


- 


- 


- 


7.12 


0 


0 


c 


- 


- 


- 


- 


- 


- 


7.12 


0.167 


0 


0 


0 


- 


- 


- 


- 


- 


7.12 


0/B 


0 


0 


0 


B 


- 


- 


- 


: 


7.13 


0 


0 


0 


0 


0 




- 


- 


- 


7.12 


0 


0 


0/C 


0 


0.100 


0 


C 


- 


- 


7.15 


0 


0 


0/C 


0 


0 


0 


0/C 


C 


!' 


7.16 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


B 
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TABLE C.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.1 


7.2 


7.3 


7.4 


7.5 


7.6 


7.29 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


7.22 


0/B 


0/B 


0/B 


0/B 


0 


0/B j 


7.29 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


7.29 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


7.29 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


7.22 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


7.23 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


7.22 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


7.25 


0/B 


0/B 


0/B 


0/B 


0 


0/B 


7.27 


0.100 


0.200 


0.273 


0 


0 


0 


7.29 


0 


0 


0 


0 


0/C 


0 


7.29 


0.077 


0.455 


0.417 


0 


0/C 


0 


7.32 


0.048 


0.038 


0.031 


0 


0/C 


0 


7.33 


0 


0 


0 


0 


0 


0.143 


7.35 


0.077 


0.333 


0.400 


0 


0/C 


0 
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TABLE C.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.7 


7.8 


7.9 


7.10 


7.11 


7.12 


7.13 


7.1 

4 


7.15 


7.16 


7.17 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


0.400/B 


7.18 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


0.400/B 


7.19 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


0.400/B 


7.20 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


0.400/B 


7.21 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


0/B 


7.2* 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


0/B 


7.23 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


0/B 


7.24 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


0/B 


7.2* 


0/B 


0/B 


0 


0 


0 


0/B 


0 


0 


0 


0/B 


7.27 


0 


0 


0 


0.07 

7 


0/D 


0 


0 


0 


0.111 


0 


7.28 


0 


0 


0 


0/C 


0 


0 


0 


o/c 


o/c 


0 


7.29 


0 


0 


0 


0.231/ 

C 


0 


0 


0 


o/c 


0.083/C 


0 


7.32 


0 


0 


0 


o/c 


0 


0 


0 


o/c 


0.050/C 


0 1 


7.33 


0.125 


0.125 


0 


0 


0.333/ 

D 


0 


0 


0 


0 


0 


7.35 


0 


0 


0 


0.200/ 

C 


0 


0 


0 


o/c 


0.091/ 

c 


0 
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TABLE C.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.17 


7.18 


7.17 


7.20 


7.2 1 1 


7.17 


B 


- 


- 


- 


! 


7.28 


0.400/B 


B 


- 


- 


_ - j 


7.29 


0.400/B 


0.400/B 


e 


- 




7.28 


0.400/B 


0.400/B 


0.400/B 


B 


- 


7.21 


0/B 


0/B 


0/B 


0/B 


e 


7.28 


0/B 


0/B 


0/B 


0/B 


0.57 l/B 


7.28 


0/B 


0/B 


0/B 


0/B 


0.57 1/B 


7.28 


0/B 


0/B 


0/B 


0/B 


0.57 l/B 


7.28 


0/B 


0/B 


0/B 


0/B 


0.429/B 


7.29 


0 


0 


0 


0 


0 


7.28 


0 


0 


0 


0 


0 


7.29 


0 


0 


0 


0 


0 


7.17 


0 


0 


0 


0 


0 


7.33 


0 


B 


0 


0 


0 


7.35 


0 


0 


0 


0 


0 
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TABLE C.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.22 


7.23 


7.24 


7.25 


7.27 


7.28 


7.29 


7.32 


7.3 

3 


7.35 


7.22 


B 


- 


- 


- 


- 




- 


- 


- 


; 


7.23 


0.571/ 

B 


B 


- 


- 


- 


- 


- 


- 


- 


- 


7.24 


0.571/ 

B 


0.571/ 

B 


B 


- 




- 


- 


- 


- 


- 


7.25 


0.429/ 

B 


0.429/ 

B 


0.429/ 

B 


B 


- 


- 


- 


- 




- 


7.27 


0 


0 


0 


0 


D 


- 


- 


- 


- 


- 


7.22 


0 


0 


0 


0 


D 


C 


- 


- 


- 


- 


7.27 


B 


0 


0 


0 


0.182 


0/C 


C 


- 


- 


- 


7.32 


0 


0 


0 


0 


0.125 


0/C 


0.037/ 

C 


C 


- 


- 


7.33 


0 


0 


0 


0 


0/D 


0 


0 


0 


D 


- 


7.35 


0 


0 


0 


0 


0.200 


0/C 


0.333/ 

C 


0.034/ 

C 


0 


C 
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TABLE C.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 



VERSION vm 





8.1 


8.2 


8.3 


8.4 


8.5 


8.6 


8.1 


8.8 


8.1 


D 


- 


- 


- 


- 


- 


- 


- 


8.2 


0/D 


D 


- 


- 


- 


- 


- 


- 


8.3 


0/D 


0.500/ 

D 


D 


- 


- 


- 


- 


- 


8.3 


0/D 


0.143/ 

D 


0.250/ 

D 


D 


- 


- 


- 


- 


8.5 


0 


1.00 


0.500 


0.429 


- 


- 


- 


- 


8.5 


0 


0.500 


0 


0.125 


0.500 


0 


- 


- 


8.2 


0 


0.333 


0.667 


0.111 


0.333 


0.667 


C 


- 


8J 


0.250/D 


0/D 


0/D 


0/D 


0 


0 


0 


D 


8.9 


0.125 


0 


0 


0 


0 


0 


0 


0 


8.11 


0 


0.333 


0.200 


0.100 


0.250 


0.200/B 


0.167 


0.667 


8.11 


0 


0 


0 


0 


0 


0/B 


0 


0 


8.13 


0.750 


0 


0 


0 


0 


0/B 


0 


0.500 


8.11 


0.222/D 


0/D 


0/D 


0/D 


0 


0 


0 


0/D 


8.15 


0.750 


0 


0 


0 


0 


0/B 


0 


1.00 


8.15 


0.143 


0 


0 


0 


0 


0/B 


0 


0 


8.11 


0.286 


0 


0 


0 


0 


0/B 


0 


0.500 


8.23 


0 


0.500 


0.333 


0.111 


0.333 


0.333/B 


0.250 


1.00 


8.25 


0 


0 


0 


0 


0 


0/B 


0 


0 


8.26 


0 


0 


0 


0 


0 


0/B 


0 


0 
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TABLE C.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION Vffl 





8.9 


8.11 


8.12 


8.13 


8.14 


8.15 


8.18 


8.19 


8.2 

3 


8.25 


8.2 

6 


8.9 


A 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


8.11 


0 


B 


- 


- 


- 


- 


- 


- 


- 


- 


- 


8.12 


0 


0/B 


B 


- 


- 


- 


- 


- 


- 


- 


- 


8.23 


0.167 


0/B 


0/B 


B 


- 


- 


- 


- 


- 


- 


- 


8.14 


0.091 


0 


0 


0.12 

5 


D 


- 


- 


- 


- 


- 


- 


8.15 


0.200 


0/B 


0/B 


1.00 


0.429 


B 


- 


- 


- 


- 


- 


8.26 


0 


0/B 


0/B 


0 


0.100 


0.25CN 

B 


B 


- 


- 


- 


- 


8.14 


0.125 


0.167/B 


0/B 


0.40 

0 


0.200 


0.75CK 

B 


CNB 


B 


- 


- 


- 


8.23 


0 


0.500/B 


0/B 


0 


0 


CNB 


(KB 


(KB 


B 


- 


- 


8.25 


0 


0/B 


0/B 


0 


0 


(KB 


(KB 


(KB 


(KB 


B 


- 


8 “ 


0 


0/B 


0/B 


0 


0 


CNB 


(KB 


(KB 


CNB 


0.66 

7\B 


B 
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TABLE C.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

version vra 



ji 


8.1 


8.2 


8.3 


8.3 


8.5 


8.6 


8.1 


8.3 


8.9 


8.27 


0 


0 


0 


0 


0 


0/B 


0 


0 


0 


8.29 


0 


0 


0 


0 


0 


0/B 


0 


0 


0 


8.29 


0 


0 


0 


0 


0 


0/B 


0 


0 


0 


8.30 


0 


0 


0 


0 


0 


0/B 


0 


0 


0 


8.33 


0 


0 


0 


0 


0 


0/B 


0 


0 


0 


8.32 


0 


0 


0 


0 


0 


0/B 


0 


0 


0 


8.33 


0 


0 


0 


0 


0 


0/B 


0 


0 


0 


8.34 


0 


0 


0 


0 


0 


0/B 


0 


0 


0 


8.30 


0.333/D 


0/D 


0/D 


0/D 


0 


0 


0 


0.143/D 


0.091 


8.37 


0.600/D 


0/D 


0 /D 


0 /D 


0 


0 


0 


0.333/D 


0.143 


8.30 


0.600 


0 


0 


0 


0 


0/B 


0 


0.333 


0.143 


8.40 


0.500 


0 


0 


0 


0 


0/B 


0 


0.250 


0.125 


8.44 


0.500 


0 


0 


0 


0 


0/B 


0 


0.333 


0.143 


8.44 


0 


0 


0.500 


0.143 


1.00 


0.500/B 


0.333 


0 


0 


8.44 


0.333 


0 


0 


0 


0 


0/B 


0 


0.167 


0.100 


8.45 


0.200 


1.00 


0.333 


0.125 


0.500 


0.333/B 


0.250 


0 


0 


8.40 


0.333 


0 


0 


0 


0 


0/B 


0.333 


0.333 


0.200 


8.48 


0.500 


0 


0.500 


0.143 


1.00 


0.500/B 


0 


0 


0 


8.50 


0 


0 


0 


0 


0 


0/B 


0 


0 


0 



97 



TABLE C.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

version vra 





8.11 


8.12 


8.13 


8.14 


8.15 


8.18 


8.19 


8.23 


8.25 


8.26 


8.27 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0.400/B 


0.500/B 


8.29 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0.667/B 


0.667/B 


8.29 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0.400/B 


0.400/B 


8.30 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.31 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.32 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.30 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.34 


0/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.45 


0 


0 


0.250 


0.333/ 

D 


0.571 


0.100 


0.300 


0 


0 


0 


8.37 


0 


0 


0.50 


0.222/ 

D 


1.00 


0.167 


0.333 


0 


0 


0 


8.38 


0/B 


0/B 


0.500/ 

B 


0.222 


1.00/B 


0.167/ 

B 


0.33/ 

B 


0/B 


0/B 


0/B 


8.44 


0/B 


0/B 


0.400/ 

B 


0.30 

0 


1.00/B 


0.143/ 

B 


0.429 

/B 


0/B 


0/B 


0/B 


8.44 


0/B 


0/B 


0.500/ 

B 


0.33 

3 


0/B 


0.167/ 

B 


0.500 

/B 


0/B 


0/B 


0/B 


8.42 


0.333/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0.500 

IB 


0/B 


0/B 


8.44 


0/B 


0/B 


0.286/ 

B 


0.25 

0 


0.667/ 

B 


0.111/ 

B 


0.429 

/B 


0/B 


0/B 


0/B 


8.45 


0.250/B 


0/B 


0/B 


0.125 


0.500/ 

B 


0.250/ 

B 


0/B 


0.33/ 

B 


0/B 


0/B 


8.46 


0/B 


0/B 


0.500/ 

B 


0.333 


1.00/B 


0/B 


0.500 

/B 


0/B 


0/B 


0/B 


8.45 


0.333/B 


0/B 


0/B 


0 


0/B 


0/B 


0/B 


0.500 

/ B 


0/B 


0/B 


8.50 


0/B 


0/B 


0/B 


0 


0/B 


i 0/B 


0/B 


0/B 


0/B 


0/B 
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TABLE C.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

version vm 





8.27 


8.28 


8.29 


8.30 


8.31 


8.31 


8.33 


8.27 


B 


- 


- 


- 


- 


- 


- 


8.29 


0.500/B 


B 


- 


- 


- 


- 


- 


8.29 


0.333/B 


0.400/B 


B 


- 


- 


- 


- 


8.30 


0/B 


0/B 


0/B 


B 


- 


- 


- 


8.37 


0/B 


0/B 


0/B 


0.444/B 


B 


- 


- 


8.32 


0/B 


0/B 


0/B 


0.667/B 


0.500/B 


B 


- 


8.30 


0/B 


0/B 


0/B 


0.667/B 


0.500/B 


0.400/B 


B 


8.34 


0/B 


0/B 


0/B 


0.364/B 


0.308/B 


0.400/B 


0.400/B 


8.30 


0 


D 


B 


B 


0 


0 


0 


8.37 


I) 


D 


D 


B 


0 


0 


0 


8.40 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.40 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.41 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.42 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.34 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.40 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.46 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.48 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


8.50 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 


0/B 
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TABLE C.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

version vra 





8.34 


8.36 


8.37 


8.38 


8.44 


8.44 


8.44 


8.44 


8.45 


8.44 


8.44 


8.5 

0 


8.3 

7 


B 




















- 


- 


8.3 

6 


0/B 


B 




















- 


8.3 

7 


D 


0.429 

/D 


D 




















8.3 

7 


0/B 


0.429 


1.00 


B 


















8.4 

1 


0/B 


0.500 


0.250 


0.750 
/ B 


B 


- 


- 


- 


- 


- 


- 


- 


8.4 

1 


0/B 


0.444 


0.600 


0.600 

/B 


0.833 

/B 


B 


- 


- 


- 


- 


- 


- 


8.4 

2 


0/B 


0 


0 


0/B 


0/B 


0/ 

B 


B 


- 




- 


- 


- 


8.4 

2 


0/B 


0.364 


0.375 


0.375 

/B 


0.444 
/ B 


0.500 

IB 


0/ 

B 


B 


- 


- 


- 


- 


8.4 

5 


0/B 


0.125 


0.250 


0.250 

/B 


0.200 

/B 


0.250 

/B 


•.600 

/B 


0.143 

IB 


B 


- 


- 


- 


8.4 

6 


0/B 


0.333 


0.400 


0.400 

/B 


0.500 

IB 


0.600 

IB 


0/ 

B 


0.375 
I B 


0.667 

IB 


B 


- 


- 


8.4 

8 


0/B 


0 


0 


0/B 


0/B 


0/ 

B 


0/ 

B 


0/ 

B 


1.00/ 

B 


0/ 

B 


B 


- 


8.5 

0 

1 1 


0.083 

/B 


0 


0 


0/B 


0/B 


0/ 

B 


0/ 

B 


0/ 

B 


0/ 

B 1 


0/ 

B 


0/ 

B 


B 1 
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APPENDIX D: FUNCTIONAL FAULT TYPE DISTRIBUTION 



In this Appendix we present (Tables D.l to D.8) the relative frequency 
of occurrence of functional fault types in the eight versions of CONFLICT 
(Shimeall 1991b). The fault types are classified in types O, I, n, HI, IV, V 
and VI according to the discussion in Section B.7 of Chapter HI . 

Some of the faults included in the failure region library by Shimeall 
(1991a) do not have a specific location and, therefore, cannot be assigned to 
any specific program module. However, it is possible by their description to 
determine which part of the CONFLICT function each of them affects. These 
faults occupy the generic entries (labeled as Others, movement,., etc) in bold 
type that precede each functional group of program modules. 

Faults that have been assigned to multiple functional groups contribute 
to the fault count of each group by an appropriate fraction of fault. For 
example, a fault distributed in three groups, will contribute to the fault count 
of each group by 1/3. 

The observed fault type frequencies are compared with the expected 
frequencies of a simple model that assumes uniform distribution of faults as 
in Appendix B. 
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TABLE D.l FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION I 10 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Others 


0 


561 


- 


8/26 


6.2 


0.24 


Conflict 


0 


94 


- 


- 


- 


- 


Ceiling 


o 


18 


- 


- 


- 


- 


Mini 


0 


7 


- 


- 


- 


- 


MinR 


o 


7 


- 


- 


- 


- 


MaxIR 


o 


7 


- 


- 


- 


- 


Outside Range 


o 


17 


- 


- 


- 


- 


Verify Input 


o 


20 


1.17 


- 


- 


- 


Check Params 


0 


32 


1.7, 1.6 


- 


- 


- 


Check Army 
Values 


0 


115 


1.10, 1.11 


- 


- 


- 


Prepare Output 


o 


49 


1.23, 1.1 


- 


- 


- 


Initialization 


o 


114 


- 


- 


- 


- 


Update Army 
Values 


0 


24 


- 


- 


- 




Update 


0 


71 


1.2 


* 


- 


- 



10 The fault rate is 26/2352 = 0.01105 faults per line (including data 
declarations in the line count) 
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TABLE D.l FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION I 



1 Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Movement 


I 


160 


- 


1/26 


1.77 


0.068 


BatVelocV 


I 


23 


- 


- 


- 


- 


AltitudeZ 


I 


30 


- 


- 


- 


- 


DistD 


I 


4 


- 


- 


- 


- 


Position 


I 


72 


1.8 


- 


- 


- 


Movement 


I 


31 


- 


- 


- 


- 


Observation 


II 


240 


- 


3/26 


2.65 


0.10 


SizeListLoc 


II 


14 


- 


- 


- 


- 


HeightH 


II 


20 


- 


- 


- 


- 


FindAngle 


II 


21 


- 


- 


- 


- 


First Condi- 


II 


41 


- 


- 


- 


- 


tion 














Second 


II 


25 


- 


- 


_ 


_ 


Condition 














Visual 


II 


10 


- 


_ 


_ 


. 


Contrast 














ObservJam 


II 


18 


1.27 




- 


• 


Third 


II 


27 


- 


. 


_ 


_ 


Condition 














Observation 


II 


64 


1.14, 1.26 


- 


- 


- 
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TABLE D.l FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION I 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Attrition 


III 


355 


- 


4/26 


3.92 


0.151 


Attrition 


III 


10 


1.13 


- 


- 


- 


Set Fire 
Upon 
Coonds 


III 


44 


1.13, 1.16. 1.15 




- 


- 


Assign LL 
Coords 


III 


21 


- 


- 


- 


- 


KilledK 


III 


46 


- 


- 


- 


- 


Num Killers 
NK 


III 


28 


- 


- 


- 


- 


Killers 

AvailKA 


m 


30 


- 


- 


- 


- 


Times 
Killers 
Used KU 


hi 


40 


* 


- 


- 


- 


Total Weap 
In Use NW 


hi 


51 


1.20 


- 


- 


- 


ClearDead 

Squads 


iii 


30 


- 


- 


- 


- 


For Each 
Weap 


hi 


30 


- 


- 


- 


- 


Communica 

tion 


IV 


678 


- 


3/26 


7.49 


0.288 


Check 

ComMsg 


IV 


21 


- 


- 


- 


- 


Send 

Reports 


IV 


47 


- 


- 


- 


- 
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TABLE D.l FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION I 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Communica 

tion 


IV 


10 


- 


- 




- 


Total 

Squads 

SendingNS 


IV 


22 


“ 


- 


- 


- 


TotalSquads 

ReceivNR 


IV 


22 


* 


- 


- 


- 


TotalSquads 

Jamming 


IV 


22 


- 


- 


- 


- 


TotalSquads 

Processing 

NR 


IV 


22 


■ 


- 


" 


- 


PutlntoList 


IV 


42 


1.22 


- 


- 


- 


SendMsgs 


IV 


121 


- 


- 


- 


- 


Process 

Command 

Messages 


IV 


29 


■ 


- 


- 


- 


Process 

Report 

Messages 


IV 


69 


■ 


- 


- 


- 


Msg 

Receipt 

DelayRd 


IV 


52 


■ 


- 


- 


- 


ManipProce 
ss List 


IV 


78 


- 


- 


- 


- 


ManipMsg 

Queue 


IV 


33 


- 


- 


- 


- 
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TABLE D.I FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION I 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


PutMsg On 
Sent LL 


IV 


22 


- 


- 


- 


- 


Instantiate 

CommandMsg 


IV 


66 


1 . 25 , 1.19 


- 


- 


- 


Environment 


V 


143 


- 


4/26 


1.58 


0.061 


Chech Weather 


V 


25 


1 . 3 , 1.4 


- 


- 


- 


TerrMove TM 


V 


25 


1 . 18 , 1.12 


- 


- 


- 


Weather 
SevFactor WF 


V 


22 


- 


- 


- 


- 


Weather 

ObservWO 


V 


12 


- 


- 


- 


- 


Weather 

MoveWM 


V 


12 


- 


- 


- 


- 


Slope Intensity 
IS 


V 


24 


- 


- 


- 


- 


Altitude 
Intensity IA 


V 


7 


- 


- 


- 


- 


IntensityLocIL 


V 


6 


- 


- 


- 


- 


Location 
Intensity BI 


V 


9 


- 


- 


- 


- 
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TABLE D.l FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION I 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Restoration 


VI 


215 


- 


3/26 


2.38 


0.092 


Squad Alive 


VI 


4 


1.9 


• 


- 


- 


Bat Alive 


VI 


16 


- 


- 


- 


- 


Restoration 


VI 


8 


- 


- 


- 


- 


TotalRestore 

dCasualtiesF 


VI 


36 


- 


- 


- 


- 


F 














Coefficient 


VI 


28 


- 


- 


- 


- 


MunSquads 


VI 


19 


- 


- 


- 


- 


Restoring 

NF 














Restore 


VI 


23 


- 


- 


- 


- 


SuppAmtFS 














Restore 


VI 


54 


1 . 5 , 1.28 


- 


_ 


1 


FactoriF 














CalcEndurE 


VI 


27 


- 


- 


- 


- ll 
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TABLE D.2 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION n 11 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative | 
Frequency 


Others 


O 


184 


- 


0/26 


3.11 


0.119 


Conflict 


0 


96 


- 


- 


- 


- 


MinReal 


0 


7 


- 


- 


- 


- 


Minlnt 


o 


7 


- 


- 


- 


- 


Max 


o 


7 


- 


- 


- 


- 


Maxlnt 


0 


7 


- 


- 


- 


- 


Roof 


o 


11 


- 


- 


- 


- 


Floor 


0 


9 


- 


- 


- 


- 


Output 


o 


9 


- 


- 


- 


- 


Simulation 


o 


17 


- 


- 


- 


- 


Initializatio 

n 


0 


18 


- 


- 


- 


- 


Movement 


I 


243 


2 . 10 , 2 . 11 , 2.13 


3/26 


4.10 


0.158 


TMove 


I 


25 


- 


- 


- 


- 


ScaleSqua 

d 


I 


67 


- 


- 


- 


- 


Positioning 


I 


16 


- 


- 


- 


- 


Velocity 


I 


21 


- 


- 


- 


- 



The fault rate is 26/1540 = 0.0169 faults per line (including data 
declarations in the line count) 
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TABLE D.2 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION n 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Xmove 


I 


9 


- 


- 


- 


- 


Ymove 


I 


9 


- 


- 


- 


- 


Movement 


I 


34 


- 


- 


- 


- 


Poslnit 


I 


14 


- 


- 


- 


- 


Movelnit 


I 


11 




- 


- 


- 


MoveOut 


I 


34 


- 


- 


- 


- 


Observation 


II 


227 


2 . 15 , 2.26 


5/26 


3.832 


0.147 


Angle Big 
Enough 


II 


60 


2 . 7 , 2.8 


- 


- 


- 


Slope 


II 


9 


- 


- 


- 


- 


Can J Seek 


II 


9 


- 


- 


- 


- 


Observation 


II 


50 


- 


- 


- 


- 


Calc 

Contrast 


n 


20 


- 


- 


- 


- 


FindPt 


ii 


9 


2.12 


- 


- 


- 


No 

Obstacles 


ii 


20 


- 


- 


- 


- 


NotObs 

Jammed 


ii 


20 


- 


- 


- 


- 


OJamming 


ii 


25 


- 


- 


- 


- 


Obslnit 


ii 


10 


- 


- 


- 


- 
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TABLE D. 2 FAULT DISTRIBUTION TO FUNCTIONAL MODULES 

IN VERSION H 



I 

Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Attrition 


III 


287 


- 


0/26 


4.85 


0.186 


Attrition 


III 


5 


- 


- 


- 


- 


Attritlnflict 


III 


9 


- 


- 


- 


- 


Weapons 


III 


63 


- 


- 


- 


- 


FireCoord 


III 


56 


- 


- 


- 


- 


Suffering 


III 


94 


- 


- 


- 


- 


Attritlnit 


III 


35 


- 


- 


- 


- 


AttritOut 


III 


25 


- 


- 


- 


- 


Communic 

ation 


IV 


369 


2.16,2.17, 2.18, 
2.19, 2.20, 
2.21, 2.22, 2.23 


14/26 


6.23 


0.240 


Communic 


IV 


9 


2.4 , 2.25 


- 


- 


- 


ation 














Update 


IV 


59 


- 


- 


- 


- 


Comm 














AddToQ 


IV 


21 


- 


- 


- 


- 


Create 


IV 


41 


2.1 


- 


- 


| 


Reports 














Create 


IV 


28 


2.2 


- 


. 


. 


Commands 














Pull 


IV 


32 


- 




. 


_ 


FnomQ 














Relay 


IV 


55 


2.3 


- 


- 


- 


Messages 














Consume 


IV 


58 


- 


_ 


_ 


_ 


Reports 
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TABLE D. 2 FAULT DISTRIBUTION TO FUNCTIONAL MODULES 

IN VERSION H 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Consume 

Commands 


IV 


51 


2.5 


* 


- 


- 


Commlnit 


IV 


15 


- 


- 


- 


- 


Environme 

nt 


V 


141 


- 


3/26 


2.38 


0.092 


Dist 


V 


4 


- 


- 


- 


- 


Alt 


V 


25 


2 . 30 , 2 . 6, 131 


* 


- 


- 


Poslntens 


V 


31 


- 


- 


- 


- 


WTotal 


V 


27 


- 


- 


- 


- 


WMove 


V 


21 


- 


- 


- 


- 


WObs 


V 


21 


- 


- 


- 


- 


Height 


V 


12 


- 


- 


- 


- 


Restoration 


VI 


89 


2.14 


1/26 


1.50 


0.058 


Restoration 


VI 


89 


- 


- 


- 


- 
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TABLE D.3 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 



VERSION m 12 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Others 


O 


312 


- 


7.5/40 


10.4 


0.26 


Conflict 


O 


81 


3.10, 3.17, 
3.21, 3.32 


- 


- 


- 


max 


o 


5 


- 


- 


- 


- 


min 


0 


4 


- 


- 


- 


- 


Cieling 


0 


6 


- 


- 


- 


- 


Change 


o 


6 


- 


- 


- 


- 


Output 


0 


64 


- 


- 


- 


- 


DataUpdate 


0 


75 


3.15 


- 


- 


- 


Perfoim 

Passive 

Functions 


o 


5 


- 


- 


- 


- 


InitVals 


0 


63 


3.5, 3.24, 3.26 


- 


- 


- 


Movement 


f 


94 


- 


2/40 


3.13 


0.078 


BatPosition 


1 


67 


3.25, 3.6 


- 


- 


- 


positioning 


1 


8 


- 


- 


- 


- 


Move 


I 


19 


- 


- 


- 


- 



12 The fault rate is 40/1200 = 0.00333 faults per line (including data 
declarations in the line count) 
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TABLE D.3 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

version in 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Observation 


II 


147 


3.34 


3/40 


4.9 


0.123 


SubAngle 


II 


13 


3.34 




- 


- 


sighting 


n 


66 


3.34 


- 


- 


- 


Summation 


II 


13 


- 


- 


- 


- 


observe 


II 


5 


- 


- 


- 


- 


Attrition 


III 


131 


3.7, 3.8 


7/40 


4.37 


0.109 


DoDamage 


in 


9 


- 


- 


- 


- 


Weapon 

Sighting 


hi 


57 


3.9, 3.18, 3.19, 
3.44, 3.45 


- 


- 


- 


Attrition 


hi 


40 


- 


- 


- 


- 


Aggression 


hi 


16 


- 


- 


- 


- 


DoAction 


hi 


6 


- 


- 


- 


- 


Communica 

tion 


IV 


407 


3.36, 3.37, 
3.38, 3.39, 
3.40, 3.41, 
3.42, 3.35 


13.5/40 


13.57 


0.339 


InitRec 


IV 


13 


- 


- 


- 


- 


ScanQueue 


IV 


26 


- 


- 


- 


- 


PutlnQueue 


IV 


16 


3.2 


- 


- 


- 


Follow 

Command 

Messages 


IV 


132 


3.3, 3.4 
3.15 


- 


- 


- 


ScanQueue 


IV 


22 


- 


- 


- 


- 
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TABLE D.3 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION ffl 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Receive 

Messages 


IV 


58 


3.23 


- 


- 


- 


NM 


IV 


10 


3.1 


- 


- 




Compare Reed 
Messages 


IV 


46 


- 


- 


- 


- 


Send 

Observations 


IV 


31 


- 




- 


- 


Send Orders 


IV 


28 


- 


- 


- 


- 


Send Messages 


IV 


5 


- 


- 


- 


- 


Update 


IV 


10 


- 


- 


- 


- 


Environment 


V 


85 


- 


5/40 


2.833 


0.071 


Distance 


V 


4 


- 


- 


- 


- 


findA 


V 


16 


3.11, 3.20, 
3.28, 3.16 


- 


- 


- 


Altitude 


V 


6 


- 


- 


- 


- 


BI 


V 


15 


- 


- 


- 


jj 


TM 


V 


12 


3.12 


- 


- 


- 


WF 


V 


14 


- 


- 


- 


- 


WM 


V 


10 


- 


- 


- 


- 


WO 


V 


6 


- 


- 


- 


- 


Restoration 


VI 


24 


3.33 


2/40 


0.800 


0.020 


Restore 


IV 


24 


3.22 


- 


- 


1 
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TABLE D.4 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION IV 13 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Others 


0 


271 


- 


0/23 


3.124 


0.136 


Conflict 


0 


128 


- 


- 


- 


- 


Initialize 


O 


88 


- 


- 


- 


- 


Prep Output 


0 


55 


- 


- 


- 


- 


Movement 


I 


209 


- 


0/23 


2.410 


0.105 


Movement 


I 


S8 


- 


- 




- 


Calc Velocity 


I 


23 


- 


- 


- 


- 


MTerrain 


I 


28 


- 


- 


- 


- 


Position Squads 


I 


120 


- 


- 


- 


- 


Observation 


II 


266 


- 


11.5/23 


3.067 


0.133 


Observation 


II 


49 


4.6, 4.7, 
4.10, 4.12, 
4.18 


- 


- 


- 


Init Visual 


II 


16 


4.5 


- 


- 


- 


ValidObservatio 

n 


II 


13 


4.20 


- 


- 


- 


Angle 


II 


64 


- 


- 


- 


- 


Line 


II 


24 


4.9, 

4.20,4.22 


- 


- 


- 


Obs Contrast 


II 


25 


- 


- 


- 


- 


Location List 


II 


20 


4.9 


- 


- 


- 


Visual Contrast 


II 


30 


- 


- 


- 


- 



The fault rate is 23/1995=0.01153 faults per line (including data 
declarations in the line count) 
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TABLE D.4 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION IV 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


ObsJamming 


II 


25 


4.44.16, 

4.20 


- 


- 


- 


Attrition 


III 


318 


- 


2.5/23 


3.67 


0.159 


Attrition 


III 


4 


- 


- 


- 


- 


Inflict 


III 


12 


- 


- 


- 


- 


SetCoordinates 


III 


55 


4.8, 4.19 


- 


- 


- 


Weapon Count 


in 


21 


- 


- 


- 


- 


Weapon Inflict 


III 


60 


- 


- 


- 


- 


Suffer 


III 


13 


- 


- 


- 


- 


Init Fire List 


in 


60 


- 


- 


- 


- 


Squad Damage 


III 


30 


4.3 


- 


- 


- 


Is Destroyed 


III 


4 


- 


- 


- 


- 


Is Casualty 


III 


30 


- 


- 


- 


- 


Communicatio 


IV 


584 


- 


7/23 


6.733 


0.292 


ns 














Communicatio 


IV 


91 


- 


- 


- 


- 


ns 














Process Msg 


IV 


65 


4.13, 4.14, 
4.26 


- 


- 




SendMsg 


IV 


54 


- 


- 


- 


- 


QueueMsg 


IV 


90 


4.1 


- 


- 


- 


ReceiveMsg 


IV 


81 


- 


- 


- 


- 


AddToList 


IV 


50 


4.15 


- 


- 


- 


CmdReplace 


IV 


74 


4 . 11 , 4.24 


- 


- 


- 
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TABLE D.4 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION IV 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Report 

Messages 


IV 


45 


- 


- 


- 


- 


Command 

Messages 


IV 


34 


- 


- 


- 


- 


Environment 


V 


144 


- 


1/23 


1.66 


0.072 


Altitude 


V 


23 


4.2 


- 


- 


- 


Distance 


V 


36 


- 


- 


- 


- 


WE 

Observation 


V 


22 


- 


- 


- 


- 


WE Movement 


V 


21 


- 


- 


- 


- 


Slope Intensity 


V 


17 


- 


- 


- 


- 


Alt Intensity 


V 


22 


- 


- 


- 


- 


Restoration 


VI 


203 


- 


1/23 


2.340 


0.102 


Restoration 


VI 


36 


- 


- 


- 


- 


New Casualties 


VI 


14 


- 


- 


- 


- 


Total 

Casualties 


VI 


11 


- 


- 


- 


- 


Restore 

Amount 


VI 


25 


- 


- 


- 


- 


Restore 

Supplies 


VI 


18 


- 


- 


- 


- 


SquadFixers 


VI 


8 


- 


- 


- 


- 


Battalion 

Size 


VI 


19 


4.21 


- 


- 


- 


Transfer 


VI 


72 


- 


- 


- 


- 
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TABLE D.5 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION V 14 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Others 


0 


362 


- 


13/40 


9.378 


0.234 


Conflict 


0 


31 


- 


- 


- 


- 


Simulate 


0 


31 


- 


- 


- 


- 


Validate 


0 


149 


5.6, 5.7, 5.9, 
5.10, 5.11, 
5.12, 5.28, 
5.30, 532 


- 


- 


- 


Set Initial 


O 


45 


5.3, 5.14, 5.31 


- 


_ 


_ 


Values 














Output 


O 


56 


5.42 


- 


- 


- 


Results 














Movement 


I 


102 


- 


0/40 


2.642 


0.066 


Position 


I 


58 


- 


- 


- 


- 


Movement 


I 


44 


- 


- 


- 


- 


Observation 


31 


305 


5.20, 5.43, 5.37 


6/40 


7.902 


0.12S 


Observation 


II 


237 


- 


- 


- 


- 


SpacePoints 


31 


50 


5.1, 5.17 


- 


- 


- 


IntnstyLoc 


31 


18 


5.13, 5.41 


- 


- 


- 


Attrition 


III 


280 


- 


2/40 


7.254 


0.181 


Change Old 


III 


10 


- 


- 


- 


- 


Change 


III 


9 


- 


- 


- 


- 


Squad 














Attrition 


III 


11 


- 


- 


- 


- 


Suffer 


III 


60 


- 









The fault rate is 40/1544 =0.026 faults per line (including data 
declarations in the line count) 
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TABLE D.5 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION V 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Inflict 


in 


152 


5.15, 5.16 


* 


- 


- 


Wear 


IM 


38 


- 


- 


- 


- 


Com 

munication 


IV 


349 


5.2, 5.19, 

5.24.5.25.5.2 
6,5.27,5.39, 1 

5.40 

5.21.5.22.5.2 
3 


15.5/40 


9.041 


0.226 


Comnicat 


IV 


239 


5.4, 5.5, 5.8, 
5.33, 5.34 


- 


- 


- 


StoieMess 


IV 


39 


- 


- 


- 


- 


ReprtMess 


IV 


25 


- 


- 


- 


- 


Commnd 

Mess 


IV 


27 


- 


- 




- 


Update 

CommVars 


IV 


19 


- 


- 


- 


- 


Environment 


V 


88 


- 


1.5/40 


2.280 


0.057 


Distance 


V 


7 


- 


- 


- 


- 


Altitude 


V 


19 


5.13, 5.41 


- 


- 


- 


CompWeath 


V 


23 


- 


- 


- 


- 


WEffMov 


V 


12 


- 


- 


- 


- 


TEffMov 


V 


19 


- 


- 


- 


- 


WEffObs 


V 


8 


- 


- 


- 


- 


Restoration 


VI 


59 


5.18 


2/40 


1.528 


' 

0.038 


Restoration 


VI 


59 


5.35 


- 


- 


- 
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TABLE D.6 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 



VERSION VI 15 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequen 

cy 


Others 


O 


409 




4.5/20 


3.686 


0.184 


Conflict 


O 


95 


6.2 


- 


- 


- 


Min 


O 


9 


- 


- 


- 


- 


Max 


O 


9 


- 


- 


- 


- 


IMin 


O 


9 


- 


- 


- 


- 


IMax 


O 


9 


* 


- 


- 


- 


Ceiling 


O 


14 


- 


- 


- 


- 


CheckBatt 


O 


47 


- 


- 


. 


. 


Constar jj 














Init Battalion 




88 


6.8, 6.20 


- 


- 


- 


Initialize 


O 


44 


- 


- 


- 


- 


Perfom 


O 


24 


6.2, 6.4 


- 


_ 


- 


Simulation 














Perfoim OneDt 


O 


12 


* 


- 


- 


- 


Prepare For 


O 


12 


' 

6.7 








NextDt 














Determine 


O 


37 


- 


. 




_ 


Output 















The fault ra is 20/2219=0.0090 faults per line (including data 
declarations ..i the line count) 
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TABLE D.6 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION VI 



ft Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequen 

cy 


Movement 


I 


96 


- 


S/20 


0.865 


0.043 


Update 














Battalion 


I 


21 


' 


- 


- 


- 


Velocity 














AlignSquads 


I 


49 


- 


- 


- 


- 


Movement 


I 


9 


- 


- 


- 


- 


Move 


I 


17 


- 


- 


- 


- 


Battalion 














Observation 


11 


323 


- 


3/20 


2.911 


0.146 


Create 


II 


11 


- 


_ 


_ 


_ 


LOSList 














Observation 


II 


12 


- 


- 


- 


- 


GenObsList 


II 


42 


- 


- 


- 




Observable 


II 


16 


6.1 


- 


- 


- 


AngleSub 


II 


79 


- 


- 


- 


- 


Greater 














Updat 


II 


13 


6.3 


- 


- 


- 


LOSList 














LOSClear 


II 


24 


6.16 


- 


- 


- 


CntrstOk 


II 


30 


- 


- 


- 


- 


Location 


II 


30 


- 


_ 


_ 


_ 


Intensity 














Obs Jamming 


II 


28 


- 


- 


- 


- 


R SumObs 


IV 


1 38 


- 


_ 




_ 


'i ToNextBatt 















121 



TABLE D.6 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION VI 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Attrition 


III 


201 


6.23 


5/20 


1.812 


0.091 


Attrition 


III 


42 


6.14 


- 


- 


- 


Num 

Weapons 


III 


27 


6.9 


- 


- 


- 


Track 

Weapons 


III 


29 


6.5 


- 


- 


- 


UpdatUse 

List 


III 


29 


- 


- 


- 


- 


Choose 

Targets 


III 


45 


6.6 


- 


- 




Suffer 

Attrition 


III 


67 


- 


- 


- 


- 


Communic 

ations 


IV 


900 


6.24 


7.5/20 


8.112 


0.406 


Include 

CommObs 


IV 


39 


6.18 


- 


- 


- 


Collect 

Finished 

Report 


IV 


41 


- 


- 


- 


- 


UpdateLL 


IV 


42 


6.14 


- 


- 


- 


Communic 

ation 


IV 


16 


- 


- 


- 


- 


Send 

Communic 

ations 


IV 


35 


- 


- 


- 


- 


Send 

Report 


IV 


35 


6.2 


- 


- 


- 
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TABLE D.6 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION VI 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequen 

cy 


NewNum 

Send 


IV 


28 


- 


- 


- 


- 


Send 

Command 


IV 


29 


- 


- 


- 


- 


ReceiveCom- 

munications 


IV 


11 


- 


- 


- 


- 


Find Receiv- 
ing Delay 


IV 


55 


6.12 


- 


- 


- 


Receive 

Reports 


IV 


56 


- 


- 


- 


- 


Receive 

Commands 


IV 


57 


- 


- 


- 




Update 

NumVars 


IV 


30 


- 


- 


- 


- 


ProcessCom- 

munications 


IV 


11 


- 


- 


- 


- 


Handle 

Queuing 


IV 


8 


- 


- 


- 


- 


Queue 

Reports 


IV 


38 


- 


- 


- 


- 


Find Queue 
Report 


IV 


19 


- 


- 


- 


- 


Queue 

Commands 


IV 


38 


- 


- 


- 


- 


Find Queue 
Spot 


IV 


19 


- 


- 


- 


- 


Processing 

Delay 


IV 


24 


- 


- 


- 


- 
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TABLE D.6 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION VI 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequen 

cy 


Process 

Messages 


IV 


67 


- 


- 


- 


- 


Find Next 
Report 


IV 


19 


- 


- 


- 


- 


Find Next 
Command 


IV 


20 


- 


- 


- 


- 


Take A 
Command 


IV 


13 


- 


- 


- 


- 


Take A 
Report 


IV 


13 


- 


- 


- 


- 


New Num 
Processing 


IV 


20 


- 


- 


- 


- 


Collect 

Commands 


IV 


42 


6.2, 6.11 


- 


- 


- 


Collect 

Command 


IV 


43 


- 




- 


- 


Putin 

Commands 


IV 


32 


6.13, 6.15 


* 


- 


- 


Environment 


V 


176 


- 


0/20 


1.586 


0.079 


Distance 


V 


9 


- 


- 


“I 

- —5 


L_ 
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TABLE D.6 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION VI 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Height 


V 


28 


- 


- 


- 


- 


Update 

Weather 


V 


7 


- 


- 


- 


- 


Update 

Present 

Events 


V 


41 


- 


- 


- 


- 


Add New 
Events 


V 


24 


- 


- 


- 


- 


Weather 

Severity 


V 


42 


- 


• 




- 


WEOn 

Movement 


V 


25 


- 


- 


- 


- 


Restoration 


VI 


114 


- 


0/20 


1.027 


0.051 


Restoration 


VI 


17 


- 


- 


- 


- 


NewNum 

Fixers 


VI 


24 


- 


- 


- 


- 


Apportion 

Fixing 


VI 


53 


- 


- 


- 


- 


Remove 
Destroyed 
H Squads 


VI 


20 


- 


- 


- 


- 
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TABLE D.7 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION VD 16 17 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequenc 

y 


Expected 

Faults 


Relative 

Frequen 

cy 


Others 


0 


603 


- 


4.5/31 


10.385 


0.335 


Conflict 


0 


36 


7.28 


- 


- 


- 


float 


0 


4 


- 


- 


- 


- 


WriteError 


0 


20 


- 


- 


- 


- 


Process 


0 


107 


- 


- 


- 


- 


Check Params 


0 


34 


- 


- 


- 


- 


Check NArmy 


0 


13 


- 


- 


- 


- 


CheckPer 

Battalion 


O 


97 


7.6, 7.7, 7.8, 
7.14 


- 


- 


- 


CheckPerSquad 


0 


21 


- 


- 


- 


- 


CheckPer Enemy 


0 


15 


- 


- 


- 


- 


Check Per 
Weapon 


0 


32 


- 


- 


- 


- 


Makelnt 


0 


7 


- 


- 


- 


- 


Initialize 


0 


54 


7.28 


- 


- 


- 


SetSuad 


0 


28 


- 


- 


- 


- 


Update Info 


o 


37 


- 


- 


- 


- 


Update Battalion 


o 


42 


7.28 


- 


- 


- 



16 The fault rate is 31/1800=0.0172 faults per line (including data 
declarations in the line count) 

In this Version, certain sub procedures are declared in more than one 
locations 
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TABLE D7. FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION vn 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


Min 


O 


6 


- 


- 


- 


- 


Min 


0 


5 


- 


- 


- 


- 


SetOutput 


0 


24 


- 


- 


- 


- 


Greatest 


0 


6 


- 


- 


- 


- 


Least 


0 


6 


- 


- 


- 


- 


CheckBatta 

lionlnfo 


0 


9 


- 


- 




- 


Movement 


I 


289 


- 


0/31 


4.977 


0.161 


Invalid 

Position 


I 


10 


- 


** 


- 


- 


Velocity 


I 


20 


- 


- 


- 


- 


SetSquad 


I 


28 


- 


- 


- 


- 


Position 

Squadrons 


I 


19 


- 


- 


- 


- 


SetPosition 


I 


48 


- 


- 


- 


- 


Velocity 


I 


20 


- 


- 


- 


- 


SetPosition 


I 


48 


- 


- 


- 


- 


Update 

Position 


I 


24 


- 


- 


- 


- 


Movement 


I 


19 


- 


- 


- 


- 


Velocity 


I 


20 


- 


- 


- 


- 


Get 

Difference 


I 


33 


- 


- 


- 


- 
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TABLE D.7 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION vn 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequen 

cy 


Observation 


II 


268 


7.17, 7.35 


7/31 


4.616 


0.149 


Observation 


II 


32 


7.5, 7.10, 7.13 


- 


- 


- 


Visible Squad 


II 


40 


7.1 


- 


- 


- 


SubAngle 


II 


27 


- 


- 


- 


- 


GetAngle 


II 


40 


- 


- 


- 


- 


Series 


II 


17 


7.32 


- 


- 


- 


ClearView 


II 


31 


- 


- 


- 


- 


OContrast 


II 


35 


- 


- 


- 


- 


Intensity 


II 


18 


- 


- 


- 


- 


0 Jamming 


II 


IS 


- 


- 


- 


- 


Calc Num 


II 


43 


- 


- 


_ 


_ 


Observ 














Attrition 


III 


221 


- 


5/31 


3.806 


0.123 


Initialize 


III 


45 


7.15 


- 


_ 




WeapData 














Initialize 


III 


45 


- 


- 


- 


_ 


WeapData 














Attrition 


III 


13 


- 


- 


- 


- 


Inflict 


III 


45 


7.2, 7.3, 7.29 


- 


- 


_ 


| Attrition 














| SetFire 


III 


28 


- 


- 


- 


, 
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TABLE D7. FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION vn 



II Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 1 
Frequency 


CalcNum 


m 


48 


7.27 


- 


_ 


. 


WeapToUse 














Calculate 


III 


31 


- 


_ 


- 


_ 


Damages 














Set Damage 


III 


9 


- 


- 


- 


- 


Get Status 


III 


17 


- 


- 


- 


- 


Com 

munication 


IV 


181 


7.18, 7.9, 
7.19, 

7.20,7.21, 
7.22, 7.23, 
7.24,7.25 


10.5/31 


3.117 


0.101 


Check 


IV 


21 


- 


_ 


- 


_ 


Messages 














Command 


IV 


80 


7.4 


- 


_ 


- 


Mess 














RecDelay 


IV 


14 


- 


- 


- 


- 


Jammed 


IV 


26 


- 


_ 


_ 


_ 


Squads 














Makelnt 


IV 


7 


- 


- 


- 


- 


Incorporate 


IV 


33 


7.28 


- 


- 


- 


Environment 


V 


141 


- 


2/31 


2.428 


0.078 


Check 


V 


48 


- 


_ 


_ 


_ 


Weather 














GetTs 


V 


7 


7.11, 7.33 


- 


- 


- 


Altitude 


V 


18 


- 


- 


- 


- 


Distance 


V 


8 


- 


- 


- 


- 


WFactor 


V 


21 


- 


- 


- 


- 
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TABLE D.7 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION vn 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


WXPosition 


V 


5 


- 


- 


- 


- 


WYPosition 


V 


5 


- 


- 


- 


- 


Height 


V 


10 


- 


- 


- 


i 


WObserve 


V 


13 


- 


- 


- 


- 


W 

Movement 


V 


12 


- 


- 


- 


- 


TenEffect 


V 


15 


- 


- 


- 


- 


Distance 


V 


8 


- 


- 


- 


- 


Restoration 


VI 


97 


7.16 


2/31 


2.0 


0.065 


DeltaFixSup 


VI 


13 


- 


- 


- 


- 


Set 

Restoration 


VI 


23 


7.12 


- 


- 


- 


Change 
Squad Data 


VI 


61 


- 


- 


- 


- 
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TABLE D.8 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION Vffl 18 



| Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 
Frequency || 


Others 


0 


258 


- 


0/41 


7.778 


0.190 


Conflict 


0 


93 


- 


- 


- 


- 


UpdateU 


o 


14 


- 


- 


- 


- 


SetKU 


0 


15 


- 


- 


- 


- 


Init 

Variables 


0 


47 


- 


- 


- 


- 


Length Of 
List 


0 


11 


- 


- 


- 


- 


Update Vars 


0 


53 


- 


- 


- 


- 


Update 

Nums 


0 


4 


- 


- 




- 


Prepare 

Output 


0 


12 


- 


- 




- 


Ceiling 


o 


S 


- 


- 


- 


- 


Movement 


I 


138 


8.18, 8.35 


4/41 


4.160 


0.101 


SquadPos 


I 


54 


8.2 


- 


- 


- 


Velocity 


I 


25 


- 


- 


- 


- 


Movement 


I 


14 


- 


- 


- 


- 



The fault rate is 41/1366 =0.030 faults per line (including data 
declarations in the line count) 
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TABLE D.8 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION vm 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


SetLocation 


I 


45 


8.23 


- 


- 


- 


Observation 


n 


203 


8.15, 8.19, 8.26 


11/41 


5.939 


0.145 


VisContrast 


U 


7 


8.36 


- 


- 


- 


Observation 


II 


60 


8.9 


- 


- 


- 


Height 


II 


6 


8.41 


- 


- 


- 


FindAngle 


II 


37 


- 


- 


- 


- 


CalcAngle 


II 


11 


8.37, 8.38 


- 


- 


- 


FindPoints 


II 


17 


8.39, 8.40 


- 


- 


- 


SumObsJam 


II 


25 


8.14 


- 


- 


- 


Observable 


II 


26 


- 


- 


- 


- 


ResetObser 
ve Lists 


II 


14 


- 


- 


- 


- 


Attrition 


III 


211 


- 


3/41 


6.361 


0.155 


Attrition 


m 


30 


8.13 


- 


- 


- 


NumOf 

Weapons 


hi 


15 


- 


- 


- 


- 


SetAttacked 


in 


19 


- 


- 


- 


- 


UpdateK 


hi 


29 


8.48 


- 


- 


- 


CalcCas 


hi 


21 


- 


- 


- 


- 


CalcBK 


hi 


22 


- 


- 


- 


- 
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TABLE D.8 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

VERSION vm 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


UpdateKA 


III 


2* 


8.1 


- 


- 


- 


UpdateKU 


m 


29 


- 


- 


- 


- 


ClearAttack 

Lists 


III 


14 


- 


• 


- 


- 


SetStatus 


III 


17 


- 


- 


- 


- 


Communica 

tion 


IV 


358 


8.27, 8.28, 
8.29, 8.30, 
8.31, 8.32, 
8.33, 8.34 


15/41 


10.793 


0.263 


1 InsertMsg 


IV 


41 


8.5 


- 


- 


- 


Command 
i Msg 


IV 


13 


- 


- 


- 


- 


InsertCom 


IV 


17 


- 


- 


- 


- 


ReportMsg 


IV 


7 


- 


- 


- 


- 


, InsertRep 


IV 


29 


- 


- 


- 


- 


Communica 
! tion 


IV 


7 


- 


- 




- 


Receive 

Delay 


IV 


31 


- 


- 


- 


- 


CalRDelay 


IV 


29 


8.12, 8.50 


- 


- 


- 


QueDelay 


IV 


27 


- 


- 


- 


- 


PutQue 


IV 


39 


8.3, 8.4, 8.6 


- 


- 


- 


ProcessQue 


IV 


28 


- 


- 


- 


- 


ProcessMsg 


IV 


39 


- 


- 


- 


- 


MergeRep 

Msg 


IV 


12 


- 




- 
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TABLE D.8 FAULT DISTRIBUTION TO FUNCTIONAL MODULES IN 

version vm 



Procedure 


Type 


Lines 


Observed 

Faults 


Relative 

Frequency 


Expected 

Faults 


Relative 

Frequency 


MergeCom 

Msg 


IV 


39 


8.7 


- 


- 


- 


Environment 


V 


110 


- 


7/41 


3.316 


0.081 


Linear 

Distance 


V 


7 


- 


- 


- 


- 


Altitude 


V 


19 


8.42 


- 


- 


- 


WSevFactor 


V 


20 


- 


- 


- 




CalcBI 


V 


24 


8 . 8 , 8 . 10 , 
8 . 11 , 8.44 


- 


- 


- 


Terrain Effect 


V 


21 


- 


- 


- 


- 


Weather 
Move Effect 


V 


13 


8.45 


- 


- 


' 


WObsEffect 


V 


6 


8.46 


- 


- 


- 


Restoration 


VI 


88 


8.25 


1/41 


2.653 


0.065 


UpdateFS 


VI 


18 


- 


- 


- 


- 


Update FF 


VI 


15 


- 


- 


- 


- 


UpdateE 


VI 


55 


* 


- 


- 


- 
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APPENDIX E: SHARED BOUNDING DIMENSIONS IN FUNCTION- 



ALLY CLUSTERED FAILURE REGIONS 

In this Appendix we present a detailed analysis, whether functional 
clustering of failure regions results in an increased number of shared 19 
bounding dimensions. 

The results, are presented in the tables El to E8. Each of the tables, 
corresponds to one of the eight versions of the CONFLICT program 
(Shimeall 1991, 1991b). Each table entry, corresponds to a pair of failure 
regions of the same version as in Appendix C. 

Table entries on the main diagonal, contain the functional cluster 
identifier (I, II, III, IV, V, VI, O), for the corresponding failure region (cf. 
Chapter III). Each of the off diagonal entries contains the Jaccard coefficient 
for the identical dimensions of the two failure regions labeling the row and 
column, and the fault type identifier, in case the regions belong to the same 
functional cluster. 



In this analysis, shared dimensions are the ones that correspond to the same 
predicates, identical. 
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For example, entry (1.1, 1.6) contains 0/0 which means that the Jaccard 
Coefficient for these failure regions is equal to 0, and that both correspond 
to faults of type O. On the other hand, entry (1.5, 1.12) contains 0.154, so 
the two failure regions have a Jaccard coefficient equal to 0.154, but 
correspond to different types of faults (VI and V respectively, in this case). 
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TABLE E.l : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION I 





1.1 


1.1 


1.3 


1.4 


1.5 


1.6 


1.2 


0/0 


0 


- 


- 


- 


- | 


1.4 


0 


0 


V 


- 


- 




1.4 


0 


0 


1.333 /V 


V 


- 


- 


1.5 


0 


0.100 


0 


0 


VI 


- 


1.4 


0/0 


0/0 


0 


0 


0 


O 


1.7 


0/0 


0/0 


0 


0 


0 


0/0 


1.4 


0 


0.083 


0 


0 


0 


0 


1.7 


0 


0 


0 


0 


0 /VI 


0/0 


1.10 


0.333/0 


0/0 


0 


0 


0 


0 


1.11 


0.333/0 


0/0 


0 


0 


0 


0/0 


1.12 


0 


0.0833 


o/v 


o/v 


0.154 


o 


1.13 


0 


0 


0 


0 


0 


0 


1.14 


0 


0 


0 


0 


0 


0 
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\ 

TABLE E.l : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION I 





1.7 


1.8 


1.9 


1.10 


1.11 


1.12 


“ 

1.13 


1.7 


0 


- 


- 


- 


- 


- 


- 


1.8 


0 


I 


- 


- 


- 


- 


- 


1.9 


0 


0.063 


VI 


- 


- 


- 


- 


1.11 


0/0 


0 


0 


0 


- 


- 


- 


1.11 


0 /o 


0 


0 


0.580 /O 


0 


- 


- 


1.11 


0 


0 


0 


0 


0 


V 


- 


1.13 


0 


0.182 


0.083 


0 


0 


0 


m 


1.14 


0 


0.067 


0 


0 


0 


0 


0.143/ 

,_!!!_ 



TABLE E.l : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION I 





1.1 


1.2 


1.3 


1.4 


1.5 


1.6 


1.7 


1.17 


0.333/0 


0/0 


0 


0 


0 


0/0 


0/0 . 


i 1.28 


0 


0.100 


o/v 


o/v 


0.083 


0 


0 


1.20 


0 


0 


0 


0 


0.250 


0 


0 


1.20 


0 


0.056 


0 


0 


0.053 


0 


0 


1.17 


0 


0 


0 


0 


0 


0 


0 


1.20 


0/0 


0/0 


0 


0 


0 


0/0 


0/0 


1.2# 


0 


0 


0 


0 


0.013 


0 


0 


1.26 


0 


0 


0 


0 


0 


0 


0 


1.27 


0 


0 


0 


0 


0 


0 


0 


1.28 


0 


0 


0 


0 


0.500 /VI 


0 


0 
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TABLE E.l : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION I 





1.8 


1.9 


1.10 


1.11 


1.12 


1.13 


1.14 


1.19 


0 


0 


0.50/0 


0.50/0 


0 


0 


0 


1.19 


0 


0 


0 


0 


0.25CKV 


0 


0.111 


1.19 


0 


0.154 


0 


0 


0.100 


0.125 


0.067 


1.20 


0.118 


0.048 


0 


0 


0.053 


0.133/1 II 


0.105 


1.22 


0.056 


0.600 


0 


0 


0 


0.071 


0.048 


1.21 


0 


0 


0/0 


0/0 


0.700 


0 


0 


1.25 


0.050 


0.045 


0 


0 


0 


0.118 


0.109 


1.19 


0 


0 


0 


0 


0 


0 


0/11 


1.27 


0.120 


0.036 


0 


0 


0 


0.143/1 II 


0.074 

/II 


1.28 


0 


o/vi 


0 


0 


0.143 


0 


0 
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TABLE E.l : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION I 





1.17 


1.18 


1.19 


1.20 


1 . 2 * 


0 


- 


• 


- 


1.20 


0 


V 


- 


- 


1.20 


0 


0.100 


VI 


- 


1.20 


0 


0.100 


0.118 


m 


1.20 


0 


0 


0.067 


0.043 


1.23 


0/0 


0.250 


0.091 


0.050 


1.20 


0 


0 


0.059 


0.421 


1.20 


0 


0 


0 


0 


1.27 


0 


0 


0.045 


0.103 


1.28 


0 


0.071 


0.200 


0.050 



TABLE E.l : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION I 





1.22 


1.23 


1.25 


1.26 


1.27 


1.28 


1.22 


IV 


- 


- 


- 


- 


- 


1.23 


0 


O 


- 


- 


- 


- 


1.20 


0.042/IV 


0.048 


IV 


- 


- 


- 


1.26 


0 


0 


0 


n 


0 


0 


1.22 


0.033 


0 


0.061 


0/11 


n 


- 


1.28 


0 


0.357 


0.053 


0 


0 


VI 
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TABLE E.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.1 


2.2 


2.3 


2.4 


2.5 




2.7 


IV 


- 


- 


- 


- 




2.7 


0/IV 


IV 


- 


- 


- 


- 


2.3 


0/IV 


0.091/IV 


IV 


- 


- 


- 


2.3 


0/IV 


0.09/IV 


0. 300/1 V 


IV 


- 


- 


2.5 


0/IV 


0.067/IV 


0.133/IV 


0.133/IV 


IV 


- 


2.5 


0 


0.091 


0.091 


0.083 


0.063 


- 


2.7 


0.077 


0 


0 


0 


0 


0 


2.3 


0.077 


0 


0 


0 


0 


0 


2.10 


0 


0 


0 


0 


0 


0 


2.11 


0 


0 


0 


0 


0 


0 


2.12 


0 


0 


0 


0 


0 


0 


2.13 


0 


0.111 


0.091 


0.091 


0.154 


0 


2.14 


0 


0 


0 


0 


0 


0 


2.15 


0 


0 


0 


0 


0 


0 
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TABLE E.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.7 


2.8 


2.10 


2.11 


2.12 


2.10 


2.14 


2.15 


2.8 


n 


- 


- 


- 


- 


- 


- 


- 


2.8 


0.636/II 


n 


- 


- 


- 


- 


- 


- 


2.10 


0 


0 


I 


- 


- 


- 


- 


- 


2.11 


0 


0 


0/1 


I 


- 


I 


- 


- 


2.11 


0 


O/II 


0 


0 


n 


- 


- 


- 


2.13 


0 


0 


0/1 


0/1 


0 


I 


- 


- 


2.14 


0 


0 


0 


0 


0 


0 


VI 


- 


2.15 


0/11 


O/II 


0 


0 


O/II 


0.400 


0 


n 



TABLE E.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 



I 


2.1 


2.2 


2.3 


2.6 


2.6 


2.6 


2.18 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


2.17 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


2.18 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


2.18 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


2.20 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


2.21 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


2.22 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


2.23 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 
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TABLE E.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.7 


2.8 


2.10 


2.11 


2.12 


2.13 


2.14 


2.15 


2.18 


0 


0 


0 


0 


0 


0 


0.400 


0.400 


2.17 


0 


0 


0 


0 


0 


0 


0.400 


0.400 


2.18 


0 


0 


0 


0 


0 


0 


0.400 


0.400 


2.19 


0 


0 


0 


0 


0 


0 


0 


0 


2.20 


0 


0 


0 


0 


0 


0 


0 


0 


2.21 


0 


0 


0 


0 


0 


0 


0 


0 


2.21 


0 


0 


0 


0 


0 


0 


0 


0 


2.23 


0 


0 


0 


0 


0 


0 


0 


0 


2.25 


0 


0 


0 


0 


0 


0 


0 


0 


2.20 


O/II 


O/II 


0 


0 


0 


0 


0 


0/11 


2.30 


0 


0 


0 


0 


0 


0 


0 


0 


2.31 


0 


0 


0 


0 


0 


0 


0 


0 



TABLE E.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.5 


2.2 


2.5 


2.5 


2.5 


2.0 


2.25 


cyiv 


cyiv 


cyiv 


cyiv 


cyiv 


0 


2.24 


0 


0 


0 


0 


0 


0 


2.30 


0 


0.111 


0.111 


0.100 


0.071 


0.333/V 


- 


0 


0 


0 


0 


0 


(W || 
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TABLE E.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.16 


2.18 


2.18 


2.18 


2.20 


2.18 


IV 


- 


- 


- 


- 


2.17 


0.40 


IV 


- 


- 


- 


2.18 


0/IV 


0.400/IV 


IV 


- 


- 


2.18 


0/IV 


0/IV 


0/IV 


IV 


- 


2.20 


0/IV 


0/IV 


0/IV 


0.571/IV 


IV 


2.21 


0/IV 


0/IV 


0/IV 


0.571/IV 


0.571/IV ! 


2.21 


0/IV 


0/IV 


0/IV 


0.571/IV 


0.571/IV 


2.18 


0/IV 


0/IV 


0/IV 


0.571/IV 


0.571/IV 


2.25 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


2.26 


0 


0 


0 


0 


0 


2.30 


0 


0 


0 


0 


0 


2.31 


0 


0 


0 


0 


0 



TABLE E.2 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION n 





2.28 


2.22 


2.23 


2.24 




2.30 




2.23 


IV 


- 


- 


- 




- 




2.22 


0.571 /IV 


IV 


- 


- 




- 




2.23 


0.571 /IV 


0.571 /IV 


IV 


- 




- 




2.25 


0/IV 


0/IV 


0/IV 


IV 




- 




2.26 


0 


0 


0 


0 


n 






2.30 


0 


0 


0 


0 


0 


V 




2.31 


0 


0 


0 


0 


0 


o/v 


V 
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TABLE E.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 





3.1 


3.2 


3.3 


3.4 


3.5 


3.6 


3.7 


3.8 


3.9 


3.10 


3.2 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


- 


3.2 


0.60/IV 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


3.3 


0.25 

/IV 


0.286 

/IV 


IV 




- 


- 


- 


- 


- 


- 


3.4 


0/IV 


o/iv 


0/IV 


IV 


- 


- 


- 


- 


- 


- 


3.5 


0 


0 


0 


0 


0 


- 


- 


- 


- 


- 


3.6 


0.143 


0.200 


0 


0 


0 


I 


- 


- 


- 


- 


3.2 


0.067 


0.071 


0 


0 


0 


0.07 


m 


- 


- 


- 


3.5 


0 


0 


0 


0 


0 


0 


0/III 


m 


- 


- 


3.9 


0 


0 


0 


0 


0 


0 


0.48/III 


0/III 


HI 


- 


3.10 


0 


0 


0 


0 


0- 

.17/0 


0 


0 


0 


0 


0 


3.11 


0.111 


0.125 


0.091 


0 


0 


0.13 


0.059 


0 


0 


0 


3.12 


0.111 


0.125 


0 


0 


0 


0.11 


0.188 


0 


0.087 


0 


3.13 


0 


0 


0 


0 


0 


0 


0 


0 


0.042 


0 


3.15 


0.077 

/ iv 


0.083 

/ iv 


0.364 

/IV 


0.100 

/IV 


0/0 


0 


0 


0 


0.040 


0/0 


3.16 


0.11 


0.125 


0.091 


0 


0 


0.13 


0.059 


0 


0 


0 


3.12 


0 


0 


0 


0 


0/0 


0 


0 


0 


0 


0/0 


3.18 


0.067 


0.077 


0 


0 


0 


0.09 


0.714 

/ III 


0/III 


0.429 

/III 


0 


3.11 


0 


0 


0 


0 


0 


0 


o/m 


O/III 


O/III 


0 


3.21 


0 


0 


0 


0 


0/0 


0 


0 


0 


0 


0/0 


3.12 


0.125 


0.167 


0 


0 


0 


0.20 


0.063 


0 


0.048 


0 


1 

3.23 


0.182 

/IV 


0.200 

/IV 


0.077 

/IV 


0/IV 


0.- 

0833 


0.091 


0.111 


0 


0.040 


1 

0.111 
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TABLE E.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION HI 





3.11 


3.12 


3.13 


3.15 


3.16 


3.17 


3.18 


3.19 


3.2 


3.2 


3.2 




















1 


2 


3 


3.11 


V 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


3.12 


0.111 

/V 


V 


- 




- 


- 


- 


- 


- 


- 


- 


3.13 


0 


0 


n 


- 


- 


- 


- 


- 


- 


- 


- 


3.15 


0.067 


0 


0 


O/IV 


- 


- 


- 


- 


- 


- 


- 


3.16 


0.333/V 


0.111 

/v 


0 


0.06 

7 


V 


- 


- 




- 


- 


- 


3.17 


0 


0 


0 


0/0 


0 


O 


- 


- 


- 


- 


- 


3.18 


0.063 


0.059 


0 


0.05 


0.06 


0 


m 


_ 


- 


_ 


- 










6 


3 














3.19 


0 


0 


0 


0 


0 


0 


0/III 


m 








3.21 


0 


0 


0 


0/0 


0 


0/0 


0 


0 


0 


- 


- 


3.22 


0.111 


0.100 


0 


0 


0.11 


0 


0.15 


0 


0 


VI 


. 












1 




4 










3.23 


0.077 


0.167 


0 


0.06- 


0 


0 


0.05 


0 


0 


0.8 


IV 1 










3/1 V 






3 






33 
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TABLE E.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION ffl 





3.1 


3.1 


3.3 


3.4 


3.5 


3.6 


3.7 


3.8 


3.9 


3.10 


3.24 


0.143 


0.167 


0.167 


0 


0.286- 

10 


0 


0.071 


0 


0 


0.2- 

5/0 


3.26 


0.125 


0.167 


0.250 


0 


o 


0.250 
/ 1 


0.067 


0 


0 


0 


3.26 


0 


0 


0.125 


0.125 


0.571 

IO 


o 


o 


0 


0 


0.16- 

7/0 


3.28 


0 


0.125 


0 


0.100 


0.222 


0.125 


0.063 


0 


0 


o 


1 3.32 


0 


0 


0.167 


0 


0/0 


0 


0 


0 


0 


0/0 


' 3.33 


0/IV 


0/IV 


0.143 

/IV 


0/IV 


0 


0 


0 


0 


0 


0 


3.34 


0 


0 


0.143 


0 


0 


0 


0 


0 


0 


0 


3.35 


0/IV 


0/IV 


0.143 

/IV 


0/IV 


0 


0 


0 


0 


0 


0 


3.36 


0/IV 


0/IV 


0.167 

/IV 


0/IV 


0 


0 


0 


0 


0 


0 


3.37 


0/IV 


0/IV 


0.143 

/IV 


0/IV 


0 


0 


0 


0 


0 


0 


3.38 


0.111 

/IV 


0/IV 


0/IV 


0.125 


0.222 


0 


0.067 


0 


0 


0.167 


3.39 


0/IV 


0/IV 


0/IV 


0.125 


0.222 


0 


0 


0 


0 


0.200 


3.40 


0/IV 


0/IV 


0/IV 


0.125 


0.222 


0 


0 


0 


0 


0.200 
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TABLE E.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 





3.11 


3.12 


3.13 


3.15 


3.16 


3.17 


3.17 


3.19 


3.21 


3.22 


3.2 
























3 


3.24 


0 


0 


0 


0.10 


0 


0/0 


0.07 


0 


0/0 


0 


0.1 










/ 0 






7 








82 


3.26 


0.11 


0.10 


0 


0.09 


0.11 


0 


0.08 


0 


0 


0.16 


0.0 




1 


0 




1 


1 




3 






7 


83 


3.26 


0 


0 


0 


0.09 


0 


0/0 


0 


0 


0/0 


0 


0.0 










I/O 














83 


3.28 


0.11 


0 . 11 - 


0 


0 


0.11 


0 


0.06 


0 


0 


0.11 


0.0 




1 /v 


l/v 






1/v 




3 






1 


83 


3.32 


0 


0 


0 


0.10 


0 


0/0 


0 


0 


0.33 


0 


0 










0/0 










3 /O 






3.33 


0 


0 


0 


0.09 


0 


0 


0 


0 


0.25 


0 


0 










1 










0 


/VI 




3.34 


0 


0 


0/11 


0.09 


0 


0 


0 


0 


0.25 


0 


0 J 










1 










0 






3.35 


0 


0 


0 


0.09 

1 


0 


0 


0 


0 


0.25 


0 


0 



TABLE E.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 





3.1 


3.2 


3.3 


3.4 


3.5 


3.6 


3.7 


3.8 


3.9 


3.10 


3.41 


0.071 

/IV 


O/IV 


0/1V 


0.125 


0.222 


0 


0 


0 


0 


0.200 


3.42 


o/iv 


0.111 

/IV 


0/IV 


0.126 


0.222 


0 


0 


0 


0 


0.200 


3.43 


0.091 


0 


0 


0.167 


0 


0.143 


0.118 


0 


0.042 


0 


3.44 


0 


0.083 


0 


0 


0 


0.091 


0.770 

/III 


o/in 


0.381 

/III 


0 


3.45 


0 


0 


0 


0 


0.056 


0 


0/III 


0/III 


0.529 

/III 


0.381 
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TABLE E.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 





3.11 


3.12 


3.13 


3.1 

5 


3.1 

6 


3.1 

7 


3.19 


3.18 


3.21 


3.22 


3.23 


3.36 


0 


0 


0 


0.091 

/IV 


0 


0 


0 


0 


0.250 


0 


0 /IV 


3.37 


0 


0 


0 


0.091 

/IV 


0 


0 


0 


0 


0.250 


0 


o/iv 


3.38 


0 


0 


0 


0 

/ IV 


0 


0 


0 


0.063 


0 


0 


0.08- 

3/IV 


3.39 


0 


0 


0 


0 

/IV 


0 


0 


0 


0 


0 


0 


0.08- 

3/IV 


3.41 


0 


0 


0 


0 

/IV 


0 


0 


0 


0 


0 


0 


0.08- 

3/IV 


3.41 


0 


0 


0 


0 

/IV 


0 


0 


0 


0 


0 


0 


0.08- 

3/rv 


3.41 


0 


0 


0 


0 

/ IV 


0 


0 


0 


0 


0 


0 


0.08- 

3/IV 


3.43 


0.083 


0.077 


0/11 


0 


0.08 

3 


0 


0 


0.143 


0 


0.111 


0.067 


3.44 


0.067 


0.063 


0 


0 


0.06 

7 


0 


0/III 


0.910 

/III 


0 


0.077 


0.056 


3.45 


0 


0.05 


0 


0 


0 


0 


0/III 


0.471 

m 


0 


0 


0 
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TABLE E.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 





3.24 


3.25 


3.26 


3.28 


3.34 


3.33 


3.34 


3.35 1 


3.24 


O 


I 


- 


- 


- 


- 


- 


- 


3.25 


0 


I 


- 


- 


- 


- 


- 


- 


3.26 


0.333 

/o 


0 


O 


- 


- 


- 


- 


- 


3.28 


0.125 


0.111 


0.333 


V 


- 


V 


- 


, 


3.32 


0/0 


0 


0.167 /O 


0 


O 


- 


- 


1 


3.33 


0 


0 


0.143 


0 


0.500 


VI 


- 


- 


3.34 


0 


0 


0.143 


0 


0.500 


0.200 


n 


- 


3.35 


0 


0 


0.143 


0 


0.500 


0.400 


0.400 


IV 


3.34 


0 


0 


0.143 


0 


0.500 


0.400 


0.200 


0.40/rv 


3.39 


0 


0 


0.143 


0 


0.500 


0.200 


0.400 


0.40/IV 


3.32 


0.286 


0 


0.375 


0.200 


0 


0 


0 


0/IV 


3.39 


0.143 


0 


0.250 


0.100 


0 


0 


0 


0/IV 


3.40 


0.143 


0 


0.250 


0.100 


0 


0 


0 


0/IV 


3.40 


0.143 


0 


0.250 


0.100 


0 


0 


0 


o/iv 


3.40 


0.143 


0 


0.250 


0.100 


0 


0 


0 


0/IV 


3.33 


0 


0.125 


0 


0.543 


0 


0 


0 /II 


0 


3.44 


0.077 


0.083 


0 


0.063 


1 

0 


1 

0 


0 


0 


3.45 


0.059 


0 


0.059 


0.053 


0 


0 


0 


0 
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TABLE E.3 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION m 





3.36 


3.37 


3.38 


3.39 


3.40 


3.41 


3.42 


3.43 


3.44 


3.45 


3.36 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


- 


3.37 


0.400 

/IV 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


3.42 


0/IV 


0 

/IV 


IV 


- 


- 


- 


- 


- 


- 


- 


3.39 


0/IV 


0 

/ IV 


0.28- 

6/IV 


IV 


- 


- 


- 


- 


- 


- 


3.42 


0/IV 


0 

/ IV 


0.286 

/IV 


0.500 

/IV 


rv 


- 


- 


- 


- 


- 


3.42 


0/IV 


0 

/IV 


0.286 

/IV 


0.50- 

0/IV 


0.500 

/IV 


IV 


- 


- 


- 


- 


3.42 


0/IV 


0 

/IV 


0.286 

/IV 


0.500 

/IV 


0.500 

/IV 


0.500 

/IV 


IV 


- 


- 


- 


3.43 


0 


0 


0 


0.091 


0.091 


0.091 


0.091 


n 


- 


- 


3.42 


0 


0 


0.067 


0 


0 


0 


0 


0.143 


m 


• 


3.45 


0 


0 


0.105 


0.053 


0.053 


0.053 


0.053 


0.053 


0/III 


m 
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TABLE E.4 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 



VERSION IV 





4.1 


4.2 


4.4 


4.4 


4.4 


4.4 


4.4 


4.8 


4.9 


4.10 


4.1# 


4.1 

2 j 


4.1 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


4.2 


0 


• 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 


4.1 


J 


0.118 


m 


- 


- 


- 


- 


- 


- 


- 


- 


- 


4.4 


) 


0 


0.045 


0 


- 


- 


- 


- 


- 


- 


- 


- 


4.5 


0 


0 


0.063 


0.071 

/II 


n 


- 


- 


- 


- 


- 


- 


- 


4.4 


0 


0 


0 


0/11 


O/II 


n 


- 


- 


- 


- 


- 


- 


4.7 


0 


0 


0.059 


0.071 

/II 


0.16- 

7/II 


O/II 


n 


- 


- 


- 


- 


- 


4.8 


0 


0 


0.167 

/III 


0.059 


0.125 


0 


0.111 

/II 


m 


• 


- 


- 


- 


4.8 


1 


0 


0.063 


0.07- 

l/II 


0.25- 

O/II 


o/n 


0.167 

/ n 


0.125 


n 


- 


- 


- 


4.10 


0 


0 


0.05* 


0.06- 

3/II 


O/II 


O/II 


0.125 

/II 


0.091 


0.12- 

5/II 


n 


- 


- 


4.18 


0/IV 


0.053 


0.043 


0 


0 


0 


0 


0 


0 


0 


IV 


- 


4.18 


0 


0 


0.063 


0.21- 

4/II 


0.16- 

7/n 


O/II 


0.12- 

5/II 


0.100 


0.167 

/II 


0.10- 

o/n 


«- 


n 


4.13 


0/IV 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


4.18 


0/IV 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


4.15 


0.500 


9 


0.091 


0 


0 


0 


0 


0.214 


c 


0 


0 


0 


4.16 


0 


0.071 


n 


0.18- 

8/11 


O/II 


O/II 


O/II 


n 


O/II 


O/II 


n 


0.0- 

8/n 


4.18 


0 


0.100 


0 


o/n 


O/II 


O/II 


O/II 


0 


O/II 


O/II 


0 


o/n 


4.19 


0 


0.100 


0/II1 


0/11 


O/II 


0/11 


O/II 


O/III 


O/II 


0/11 


0 


i 

o/n 


4.20 


0 


0 


0 


0/11 


O/II 


O/II 


O/II 


0 


O/II 


O/II 


0 


o/n 
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TABLE E.4 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION IV 





4.13 


4.14 


4.15 


4.16 


4.18 


4.19 


4.20 


4.2 

1 


4.2 

2 


4.24 


4.28 


4.13 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 1 


4.14 


0.800 

/IV 


IV 


- 


- 


- 


- 


- 


- 


- 


- 




4.16 


0/IV 


0/IV 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


4.16 


0 


0 


0 


n 


- 


- 


- 


- 




- 


- 


4.18 


0 


0 


0 


0.09- 

1/11 


n 


- 




- 


- 


- 


- 


4.14 


0 


0 


0 


0.07- 

l/II 


0.12- 

5/II 


n/ra 


- 


- 


- 


- 


- 


4.20 


0 


0 


0 


0/11 


0/11 


0/11 


n 










4.24 


0 


0 


0 


0.063 


0.083 


0.077 


0 


VI 


- 


- 


- 


4.22 


0 


0 


0 


0/11 


0/11 


0/11 


O/II 


0 


n 


- 


- 


4.24 


0/IV 


0/IV 


0/IV 


0.083 


0.167 


0.143 


0 


0 


0 


IV 


- 




0/1V 


0/IV 


0/IV 


0.071 


0.125 


0.111 


0 


0.077 


0 


0.60 

/IV 


rv 1 



TABLE E.4 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION IV 





4.1 


4.1 


4.1 


4.4 


4.5 


4.6 


4.4 


4.8 


4.9 


4.10 


4.11 


4.1 

2 


4.21 


0 


0.071 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


4.22 


0 


0 


0 


O/II 


O/II 


O/II 


O/II 


0 


O/II 


O/II 


0 


O/II 


4.24 


0/IV 


0.100 


0 


0 


0 


0 


0 


0 


0 


0 


0/IV 


0 


4.26 


0/IV 


0.083 


0 


0 


0 


0 


0 


0 


0 


0 


O/.IV 


0 
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TABLE E.5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 



VERSION V 





5.1 


5.2 


5.3 


5.4 


5.5 


5.6 


5.7 


5.8 


5.9 i 


5.6 


n 


- 


- 


- 


- 


- 


- 


- 




5.7 


0 


IV 


- 


- 


- 


- 


- 


- 


- 


5.3 


0 


0 


o 


- 


- 


- 


- 


- 


- 


5.4 


0 


0.053/TV 


0 


IV 


- 


- 


- 


- 


- 


5.6 


0 


0.059/IV 


0 


0.375/IV 


IV 


- 


- 


- 


- 


5.6 


0 


0 


0.167 /O 


0 


0 


0 


- 


- 


- 


5.7 


0 


0 


0/0 


0 


0 


0/0 


0 


- 


- 


5.8 


0 


0.111/IV 


0 


0.059/IV 


0.067/IV 


0 


0 


IV 


- 


5.4 


0 


0.053 


0/0 


0 


0 


0/0 


0/0 


0 


0 


5.10 


0 


0 


0/0 


0 


0 


0/0 


0/0 


0 


0.143 

10 


5.18 


0 


0 


0/0 


0 


0 


0/0 


0/0 


0 


0/0 


5.12 


0 


0 


0/0 


0 


0 


0/0 


0/0 


0 


0/0 


5.18 


0 


0.050 


0 


0.083 


0.100 


0 


0 


0.056 


0 


5.18 


0 


0 


0/0 


0 


0 


0/0 


0/0 


0 


0/0 


5.18 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.16 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.17 


0/1 1 


0 


0 


0 


0 


0 


0 


0 


0 


5.18 


0 


0 


0 


0 


0 


0 


0 


0 


0 


519 


0 


0/1 V 


0 


0/1V 


0/1V 


0 


0 


0/IV 


0 
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TABLE E.5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.12 


5.11 


5.12 


5.13 


5.14 


5.15 


5.16 


5.17 


5.18 


5.1 

7 


5.10 


0 


- 


- 


- 


- 


- 


- 


- 


- 


- 


5.11 


0/0 


0 


- 


- 


- 


- 


- 


- 


- 


- 


5.12 


0/0 


0 . 20 - 

0/0 


0 


- 


- 


- 


- 


- 


- 


- 


5.10 


0 


0 


0 


V 


- 


- 


- 


- 


- 


- 


5.10 


0/0 


0/0 


0/0 


0.8 


O 




- 




- 


- 


5.15 


0 


0 


0 


0 


0 


m 


- 


- 


- 


- 


5.16 


0 


0 


0 


0 


0 


0 . 71 - 

4/m 


m 


- 


- 




1 5.10 


0 


0 


0 


0 


0 


0 


0 


n 


- 


- 


| 5.18 


0 


0 


0 


i 0 


0 


0 


0 


0 


VI 


. 


5.19 


0 


0 


0 


0 


0 


0 


0 


0 


0 


IV 
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TABLE E.5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.1 


5.2 


5.3 


5.4 


5.6 


5.6 


5.7 


5.8 


5.20 


O/II 


0 


0 


0 


0 


0 


0 


0 


5.21 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.24 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.23 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.24 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.25 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.26 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.27 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.23 


0 


0 


0/0 


0 


0 


0/0 


0/0 


0 


5.40 


0 


0 


0/0 


0 


0 


0/0 


0/0 


0 


5.34 


0 


0 


0/0 


0 


0 


0/0 


0/0 


0 


5.32 


0 


0 


0/0 


0 


0 


0/0 


0/0 


0 


5.33 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.34 


O/II 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.35 


0 


0 


0 


0 


0 


0 


0 


o ! 


5.37 


O/II 


0 


0 


0 


0 


0 


0 


0 


5.39 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.40 


0 


0/IV 


0 


0/IV 


0/IV 


0 


0 


0/IV 


5.21 


O/II 


0 


0 


0 


0 


0 


0 


0 


5.40 


0 


0 


0/0 


0 


0 


0/0 


0/0 


0 


1 5.43 


O/II 


0 


0 


0 


0 


0 


0 


— - — 
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TABLE E.5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.9 


5.10 


5.11 


5.12 


5.13 


5.14 


5.1 

5 


5.1 

6 


5.1 

7 


5.18 


5.19 


5.26 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.26 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


O/IV 


5.22 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


O/IV 


5.23 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


O/IV 


5.22 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


O/IV 


5.25 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


O/IV 


5.26 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


O/IV 


5.22 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


O/IV 


5.22 


0/0 


0/0 


0/0 


0/0 


0 


0/0 


0 


0 


0 


0 


0 


5.34 


0/0 


0/0 


0/0 


0/0 


0 


0/0 


0 


0 


0 


0 


0 


5.31 


0/0 


0/0 


0/0 


0/0 


0 


0/0 


0.130 


0.26 

1 


0 


0 


0 


5.32 


0/0 


0/0 


0/0 


0/0 


0 


0/0 


0 


0 


0 


0 


0 


5.34 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


O/IV 


5.34 


0 


0 


0 


0 


0 


0 


0 


0 


O/II 


0 


O/IV 


5.35 


0 


0 


0 


0 


0 


0 


0.040 


0.04 

0 


0 


O/VI 


0 


5.37 


0 


0 


0 


0 


0 


0 


0 


0 


O/II 


0 


0 


5.31 


0 


0 


0 


0 


0 


0 


0.083 


0.08 

3 


0 


0 


O/IV 


5.40 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


O/IV 


5.41 


0 


0 


0 


0 


o/v 


0 


0 


0 


O/II 


0 


0 


5.42 


0/0 


0/0 


0/0 


0/0 


0 


0/0 


0 


0 


0 


0 


0 


5.43 


0 


0 


0 


0 


0 


0 


0 


0 


O/II 


0 


0 
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TABLE E.5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.19 


5.20 


5.21 


5.22 


5.23 


5.24 


5.25 


5.26 


5.27 


5.28 


0 


n 


- 


- 


- 


- 


- 


- 


- 


5.21 


0.444/IV 


0 


IV 


- 


- 


- 


- 


- 


- 


5.28 


O/IV 


0.286 


O/IV 


IV 


- 


- 


- 


- 


- 


5.23 


0.444/IV 


0 


0.444/IV 


O/IV 


IV 


- 


- 


- 


- 


5.23 


O/IV 


0.286 


O/IV 


0.286/IV 


O/IV 


IV 


- 


- 


- 


5.25 


0.444/IV 


0 


0.444/IV 


O/IV 


0.444/IV 


O/IV 


IV 


- 


- 


5.26 


O/IV 


0.286 


O/IV 


0.286/IV 


O/IV 


0.286/1 

V 


O/IV 


IV 


- 


5.27 


0.444/IV 


1) 


0.444/IV 


O/IV 


0.444/IV 


O/IV 


0.444/1 

V 


O/IV 


IV 


5.28 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.33 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.31 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.31 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.33 


O/IV 


0 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


5.33 


O/IV 


o/n 


O/IV 


0 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


5.33 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.31 


0 


O/II 


0 


0 


0 


0 


0 


0 


0 


5.39 


O/IV 


0 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


5.40 


O/IV 


0 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


O/IV 


5.41 


0 


0/11 


0 


0 


0 


0 


0 


0 


0 


5.42 


0 


0 


0 


0 


0 


0 


0 


0 


0 


5.43 


0 


0/11 


0 


0 


0 


0 


0 


0 


0 
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TABLE E.5: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION V 





5.28 


5.30 


5.31 


5.32 


5.33 


5.34 


5.3 

5 


5.3 

7 


5.3 

9 


5.4 

0 


5.4 

1 


5.4 

2 


5.28 


0 
























5.30 


0/0 


O 






















5.32 


0/0 


0/0 


0 




















5.32 


0/0 


0/0 


0/0 


O 


















5.30 


0 


0 


0 


0 


IV 


- 


- 


- 


- 


- 


- 


- 


5.34 


0 


0 


0 


0 


o/iv 


IV 

m 


- 


- 


- 


- 


- 


- 


5.35 


0 


0 


0.053 


0 


0 


0.214 


VI 


- 




- 


- 


- 


5.37 


0 


0 


0 


0 


0 


O/II 


0 


0 


- 


- 


- 


- 


5.39 


0 


0 


0.125 


0 


0 

/IV 


0 

/IV 


0.063 


0 


IV 


- 


- 


- 


5.40 


0 


0 


0 


0 


O/IV 


0/1 

V 


0 


0 


0.05- 

3/IV 


IV 


- 


- 


5.41 


0 


0 


0 


0 


0 


O/II 


0 


O/II 


0 


0 


n/v 


- 


5.42 


0/0 


0/0 


0/0 


0/0 


0 


0 


0 


0 


0 


0 


0 


0 


5.43 


0 


0 


0 


0 


0 


O/II 


0 


O/II 


0 


0 


O/II 


0 
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TABLE E.6 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 



VERSION VI 





6.1 


6.* 


6.3 


6.4 


6.5 


6.6 


6.7 


6.8 


6.2 


n 


“ 


- 


- 


- 


- 


- 


- 


6.2 


0 


O/IV 


- 


- 


- 


- 


- 


- 


6.3 


0.357 

/II 


0 


n 


- 


- 


- 


- 


- 


6.9 


0 


0/0 


0 


0 


- 


- 


- 


- 


6.6 


0 


0 


0 


0 


m 


- 


- 


- 


6.6 


0.077 


0 


0.063 


0 


O/III 


in 


- 


- 


6.9 


0.056 


0/0 


0.048 


0/0 


0 


0.091 


0 


- 


6.9 


0.167 


0.143/0 


0 


0/0 


0 


0.091 


0.50/0 


0 


6.9 


0.059 


0 


0.05 


0 


o/m 


0.1/III 


0.286 


0.286 


6.11 


0 


0.1 43/1 V 


0 


0 


0 


0 


0 


0 


6.1* 


0 


0/IV 


0 


0 


0 


0 


0 


0 


6.1* 


0 


0/IV 


0 


0 


0 


0 


0 


0 


6.1* 


0 


0/IV 


0 


0 


0 


0 


0 


0 


6.15 


0 


0.111/IV 


0 


0 


0 


0 


0 


0 


6.18 


0364/11 


0 


0.286/II 


0 


0 


0.125 


0.083 


0.250 


6.1* 


0.056 


0 


0.048 


0 


O/III 


0.083/III 


0.091 


0.091 


6.18 


0 


0/IV 


0 


0 


0 


0 


0 


0 


6.20 


0 


0/0 


0 


0/0 


0 


0 


0/0 


0/0 


6.23 


0.056 


0 


0.048 


0 


o/m 


0.091/III 


0.091 


0.091 


6.24 


0 


0/IV 


0 


0 


0 


0 


0 


0 
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TABLE E.6 : JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION VI 





6.9 


6.11 


6.12 


6.13 


6.14 


6.1 

5 


6.1 

6 


6.1 

7 


6.18 


6.20 


6.2 

3 


6.2 

4 


6.9 


m 


- 






















6.16 


0 


IV 






















6.12 


0 


0/IV 


IV 


- 


















6.13 


0 


0/IV 


0/IV 


IV 


















6.16 


0 


0/IV 


0.059 


0/IV 


IV 


- 


- 


- 


- 


- 


- 


- 


6.15 


0 


0.143 

/IV 


0.071 

/IV 


0/IV 


0/1 

V 


IV 


- 


- 


- 


- 


- 


- 


6.16 


0.071 


0 


0 


0 


0 


0 


n 


- 


- 


- 


- 


- 


6.17 


0.067 

m 


0 


0 


0 


0 


0 


0.07 

7 


m 


- 


- 


- 


- 


6.18 


0 


0/IV 


0.067 

/IV 


0/IV 


0.40- 

0/IV 


0 


0 


0 


IV 


- 


- 


- 


6.20 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


- 


- 


1 623 


0.071 

/ HI 


0 


0 


0 


0 


0 


0.07 

7 


0.73 

/III 


0 


0 


m 




6.24 


0 


0.071 

/IV 


0.118 

/IV 


0/IV 


0.27- 

8/IV 


0.063 

/IV 


0 


0 


0.46 

7/IV 


0 


0 


IV 
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TABLE E.7: JACCARD COEFFICIENTS OF FAILURE REGIONS 



OF VERSION VH 





7.1 


7.2 


7.3 


7.4 


7.5 


7.1 


7.7 


7.2 


n 


- 


- 


- 


- 


- 


- 


7.2 


0.083 


in 


- 


- 


- 


- 


- 


7.3 


0.083 


0.818/III 


m 


- 


- 


- 


- 


7.2 


0 


0 


0 


IV 


- 


- 


- 


7.5 


0. 167/11 


0 


0 


0.071 


n 


- 


- 


7.6 


0 


0 


0 


0 


0 


0 


- 


7.5 


0 


0 


0 


0 


0 


0.333/0 


o 


7.3 


0 


0 


0 


0 


0 


0.333/0 


0.500/0 


7.9 


0 


0 


0 


0.333/IV 


0 


0 


0 


7.10 


0/11 


0.231 


0.214 


0 


0/11 


0 


0 


7.11 


0 


0 


0 


0 


0 


0.200 


0.167 


7.11 


0 


0 


0 


0 


0 


0 


0 1 


7.13 


0/11 


0 


0 


0 


0/11 


0 


0 


7.14 


0 


0 


0 


0.143 


0 


0/0 


o/o : 


7.15 


0.200 


0.091/111 


0.083/1 II 


0 


0 


0 


0 


7.16 


0 


0 


0 


0 


0 


0 


o ; 
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TABLE E.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.8 


7.9 


7.10 


7.11 


7.12 


7.13 


7.14 


7.15 


7.16 


7.8 


0 


- 


- 


- 


- 


- 


- 




- 


7.9 


0 


IV 


- 


- 


- 


- 


- 


- 


- 


7.10 


0 


0 


0 


- 


- 




- 


- 


- 


7.12 


0.167 


0 


0 


V 


- 


- 


- 


- 


- 


7.12 


0 


0 


0 


0 


VI 


- 


- 


- 


- 


7.13 


0 


0 


O/II 


0 


0 


n 


- 


- 


- 


7.14 


0/0 


0 


0 


0 


0.100 


0 


O 


- 


- 


7.10 


0 


0 


0 


0 


0 


0 


0 


m 


- 


7.16 


0 


0 


0 


0 


0/V1 


0 


0 


0 


VI 
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TABLE E.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.1 


7.2 


7.3 


7.6 


7.5 


7.6 


7.27 


0/11 


0 


0 


0 


0/11 


0 


7.18 


0 


0 


0 


0/IV 


0 


0 


7.18 


0 


0 


0 


0/IV 


0 


0 


7.20 


0 


0 


0 


0/IV 


0 


0 


7.21 


0 


0 


0 


0/IV 


0 


0 


7.22 


0 


0 


0 


0/IV 


0 


0 


7.23 


0 


0 


0 


0/IV 


0 


0 


7.20 


0 


0 


0 


0/IV 


0 


0 


7 . 2 % 


0 


0 


0 


0/IV 


0 


0 


7.27 


0.100 


0.200/1 II 


0.27 3/1 II 


0 


0 


0 


7.20 


0 


0 


0 


0/IV 


0 


0/0 


7.27 


0.077 


0.455/III 


0.417/1 II 


0 


0 


0 


7.32 


0.048/II 


0.038 


0.037 


0 


0/11 


0 


7.18 


0 


0 


0 


0 


0 


0.143 1 


7.35 


0.077/11 


0.333 


0.400 


0 


0/11 


0 1 
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TABLE E.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.7 


7.8 


7.9 


7.10 


7.11 


7.12 


7.13 


7.1 

4 


7.15 


7.16 


7.19 


0 


0 


0 


O/II 


0 


0 


O/II 


0 


0 


0.400 


7.18 


0 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0.400 


7.19 


0 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0.400 


7.28 


0 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0.400 


7.21 


0 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0 


7.2* 


0 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0 


7.21 


0 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0 


7.24 


0 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0 


7.25 


0 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0 


7.27 


0 


0 


0 


0.077 


0 


0 


0 


0 


o.iii/m 


0 


7.28 


0/0 


0/0 


0/IV 


0 


0 


0 


0 


0/0 


0 


0 


7.19 


0 


0 


0 


0.231 


0 


0 


0 


0 


0.083 


0 


7.32 


0 


0 


0 


0/11 


0 


0 


O/II 


0 


0.050 


0 


7.33 


0.125 


0.125 


0 


0 


0.333 

/v 


0 


0 


0 


0 


0 


7.35 


0 


0 


0 


0 . 20 - 

0/11 


0 


0 


O/II 


0 


0.091 


0 
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TABLE E.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.17 


7.17 


7.19 


7.20 


«.* 1 


7.17 


n 


- 


- 


- 


- 


7.19 


0.400 


IV 


- 


- 


- 


7.19 


0.400 


0.400/IV 


IV 


- 


- 


7.28 


0.400 


0.400/IV 


0.400/IV 


IV 


- 


7.23 


0 


0/IV 


0/IV 


0/IV 


IV 


7.27 


0 


0/IV 


0/IV 


0/IV 


0.571/IV 


7.23 


0 


0/IV 


0/IV 


0/IV 


0.571/IV 


7.24 


0 


0/IV 


0/IV 


0/IV 


0.571/IV 


7.2* 


0 


0/IV 


0/IV 


0/IV 


0.429/1 V 


7.27 


0 


0 


0 


0 


0 


7.28 


0 


0/IV 


0/IV 


0/IV 


0/IV 


7.19 


0 


0 


0 


0 


0 


7.32 


0/11 


0 


0 


0 


0 


7.33 


0 


0 


0 


0 


0 


7.35 


0/1 1 


0 


0 


0 


0 
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TABLE E.7: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vn 





7.22 


7.23 


7.24 


7.25 


7.27 


7.28 


7.29 


7.32 


7.3 

3 


7.35 


7.22 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


- 


7.23 


0.571 

/IV 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


7.29 


0.571 

/IV 


0.571 

/IV 


IV 


- 


- 


- 


- 


- 


- 




7.25 


0.429 

/ iv 


0.429 

/IV 


0.429 

/ iv 


IV 


- 


- 


- 


- 




- 


7.22 


0 


0 


0 


0 


m 


- 


- 


- 


- 


- 


7.28 


0/IV 


0/IV 


0/IV 


0/IV 


0 


O/IV 


- 


- 


- 




7.29 


0 


0 


0 


0 


0.182 

/III 


0 


m 


- 


- 


- 


7.32 


0 


0 


0 


0 


0.125 


0 


0.037 


n 


- 


- 


7.33 


0 


0 


0 


0 


0 


0 


0 


0 


V 


- 


7.35 


0 


0 


0 


0 


0.200 


0 


0.333 


0.034 

/II 


0 


n 
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rr ‘4BLE E.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vm 





8.7 


8.2 


8.3 


8.4 


8.6 


8.6 


8.7 


8.8 


8.1 


m 


- 


- 


- 


- 


- 


- 


- 


8.1 


0 


I 


- 


- 


- 


- 


- 


- 


8.3 


0 


0.500 


IV 


- 


- 


- 


- 


- 


8.4 


0 


0.143 


0.250 

/IV 


IV 


- 


- 


- 


- 


8.5 


0 


1.00 


0.500 

/IV 


0.429 

/IV 


IV 


- 


- 


- 


8.6 


0 


0.500 


0/IV 


0.125 

/IV 


0.500 

/IV 


IV 


- 


- 


8.7 


0 


0.333 


0.667 /IV 


0.111 /IV 


0.333 /IV 


0.667 /IV 


IV 


- 


8.8 


0.250 


0 


0 


0 


0 


0 


0 


V 


8.9 


0 


0 


0 


0 


0 


0 


0 


0 


8.10 


0 


0 


0 


0 


0 


0 


0 


o/v 


8.11 


0 


0.333 


0.200 


0.100 


0.250 


0.200 


0.167 


0.667 

/V 


8.12 


0 


0 


0/IV 


0/IV 


0/IV 


0/IV 


0 


0 


8.13 


0.750 /III 


0 


0 


0 


0 


0 


0 


0.500 


8.14 


0.222 


0 


0 


0 


0 


0 


0 


0 


8.15 


0.750 


0 


0 


0 


0 


0 


0 


1.00 


8.18 


0.143 


0/1 


0 


0 


0 


0 


0 


0 


8.19 


0.286 


0 


0 


0 


0 


0 


0 


0.500 


8.23 


0 


0.500/1 


0.333 


0.111 


0.333 


0.333 


0.250 


1.00 


8.25 


0 


0 


0 


0 


0 


0 


0 


0 


8.26 


0 


0 


0 


0 


0 


0 


0 


0 
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TABLE E.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vm 





8.9 


8.10 


8.11 


8.12 


8.13 


8.19 


8.1 

5 


8.1 

5 


8.1 

5 


8.2 

3 




T |i 


8.9 


n 


- 


- 


- 


- 


- 


- 


- 


- 


- 


- 




8.12 


0 


V 


- 


- 


- 


- 


- 


- 


- 


- 


- 


. 


8.12 


0 


o/v 


V 


- 


- 


- 


- 


- 


- 


- 


- 


- 


8.12 


0 


0 


0 


IV 


- 


- 


- 


- 


- 


- 


- 


- 


8.14 


0.16 

7 


0 


(1 


0 


m 


- 




- 


- 


- 


- 


- 


8.14 


0.09 

1 /II 


0 


0 


0 


0.12 

5 


n 




- 


- 


- 


- 


- 


8.15 


0.20 

/II 


0 


0 


0 


1.00 


0.429 

/II 


n 


- 


- 


- 


- 


- 


8.12 


0 


0 


0 


0 


0 


0.100 


0.250 


I 


- 


- 


- 


- 


8.19 


0.125 

/n 


0.167 


0.167 


0 


0.400 


0.2 

/II 


0.75 

/U 


0 


C 


- 


- 


- 


8.23 


0 


0.50 


0.50 


0 


0 


0 


0 


0/1 


0 


I 


I 


- 


8.25 


0 


0 


n 


0 


0 


0 


0 


0 


0 


0 


VI 


- 


| 8.26 


<yu 


0 


0 


0 


0 


Ofll 


0 /II 


0 


0 


0 


0.667 


LjJ 
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TABLE E.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

version vm 





8.1 


8.2 


8.3 


8.4 


8.5 


8.6 


8.7 


8.1 


8.7 


8.10 


8.27 


0 


0 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


0 


0 


8.28 


0 


0 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


0 


0 


8.29 


0 


0 


0/IV 


0IV 


0/IV 


0/IV 


0/IV 


0 


0 


0 


8.30 


0 


0 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


0 


0 


8.31 


0 


0 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


0 


0 


8.32 


0 


0 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


0 


0 


8.33 


0 


0 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


0 


0 


8.34 


0 


0 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


0 


0 


8.35 


0 


0/1 


0 


0 


0 


0 


0 


0 


0 


0 


8.38 


0.333 


0 


0 


0 


0 


0 


0 


0.143 


0.091 

/II 


0 


8.37 


0.600 


0 


0 


0 


0 


0 


0 


0.333 


0.143 

/u 


0 


8.38 


0.600 


0 


0 


0 


0 


0 


0 


0.333 


0.143 

/II 


0 


8.30 


0 


0 


0 


0 


0 


0 


0 


0 


0/11 


0 


8.44 


0.600 


0 


0 


0 


0 


0 


0 


0.250 


0.125 

/II 


0 


8.41 


0.500 


0 


0 


0 


0 


0 


0 


0.333 


0.143 

/II 


0 


8.42 


0 


0 


0.500 


0.143 


1.00 


0.500 


0.333 


o/v 


0 


o/v 


8.44 


0.333 


0 


0 


0 


0 


0 


0 


0.167 

/V 


0.100 


o/v 


8.45 


0.200 


1.00 


0.333 


0.125 


0.500 


0.333 


0.250 


o/v 


0 


o/v 


8.44 


0.333 


0 


0 


0 


0 


0 


0.333 


0.333 

/V 


0.200 


o/v 


8.42 


0.5/III 


0 


0.500 


0.143 


1.00 


0.500 


0 


0 


0 


0 


8.50 


0 


0 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0 


0 


0 
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TABLE E.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

version vm 





8.11 


8.12 


8.13 


8.14 


8.15 


8.18 


8.19 


8.23 


8.25 


8.26 


8.27 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0.500 

/IV 


0.500 


8 . 2 ? 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0.667 

/IV 


0.667 


8.27 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0.400 

/IV 


0.400 


8.33 


0 


o/iv 


0 


0 


0 


0 


0 


0 


0/IV 


0 


8.33 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0/IV 


0 


8.34 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0/IV 


0 


8.33 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0/IV 


0 


8.34 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0/IV 


0 


8.35 


0 


0 


0 


0 


0 


0/1 


0 


on 


0 


0 


8.36 


0 


0 


0.250 


0.333 

fa 


0.571 

/ n 


0.1 


0.3 

/II 


0 


0 


O/II 


8.37 


0 


0 


0.50 


0.222 

/II 


i.o/n 


0.167 


0.33- 

3/II 


0 


0 


O/II 


8.38 


0 


0 


0.500 


0.222 

/II 


1.00/II 


0.167 


0.33 

fa 


0 


0 


O/II 


8.34 


0 


0 


0 


0/H 


o/n 


0 


O/II 


O/II 


0 


O/II 


8.40 


0 


0 


0.400 


0.30- 

0/11 


1.00/II 


0.143 


0.42- 

9/II 


0 


0 


O/II 


8.41 


0 


0 


0.500 


0.333 

/II 


0/11 


0.167 


0.50 

/II 


0 


0 


O/II 


8.42 


0.333/V 


0 


0 


0 


0 


0 


0 


0.50 


0 


0 


8.44 


0/V 


0 


0.286 


0.250 


0.667 


0.111 


0.429 


0 


0 


0 


8.45 


0.250/V 


0 


0 


0.125 


0.500 


0.25 


0 


0.33 


0 


0 


8.46 


0/V 


0 


0.500 


0.333 


1.00 


0 


0.500 


0 


0 


0 


8.48 


0.333 


0 


0/III 


0 


0 


0 


0 


0.500 


0 


0 


8.50 


0 


0/IV 


0 


0 


0 


0 


0 


0 


0/IV 


0 
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TABLE E.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

VERSION vm 





8.29 


8.28 


8.29 


8.30 


8.31 


8.31 


8.33 


8.27 


IV 


- 


- 


- 


- 


- 


- 


8.28 


0.500 /IV 


IV 


- 


- 


- 


- 


- 


8.29 


0.333/IV 


0.400/IV 


IV 


- 


- 


- 


- 


8.3# 


0/IV 


0/IV 


0/IV 


IV 


- 


- 


- 


8.32 


0/IV 


0/IV 


0/IV 


0.444/IV 


IV 


- 


- 


8.32 


0/IV 


0/IV 


0/IV 


0.667 /IV 


0.500 /IV 


IV 


- 


8.3# 


0/IV 


0/IV 


0/IV 


0.667 /IV 


0.500 /IV 


0. 800/1 V 


IV 


8.34 


0/IV 


0/IV 


0/IV 


0.364 /IV 


0.308/IV 


0.400/IV 


0.400 /IV 


8.34 


0 


0 


0 


0 


0 


0 


0 


8.36 


0 


0 


0 


0 


0 


0 


0 


8.37 


0 


0 


0 


0 


0 


0 


0 


8.38 


0 


0 


0 


0 


0 


0 


0 


8.39 


0 


0 


0 


0 


0 


0 


0 


8.40 


0 


0 


0 


0 


0 


0 


0 


8.4S 


0 


0 


0 


0 


0 


0 


0 


8.4# 


0 


0 


0 


0 


0 


0 


0 


8.43 


0 


0 


0 


0 


0 


0 


0 


8.45 


0 


0 


0 


0 


0 


0 


0 


8.48 


0 


0 


0 


0 


0 


0 


0 


8.48 


0 


0 


0 


0 


0 


0 


o 1 


8.50 


0 /IV 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV 


0/IV j 
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TABLE E.8: JACCARD COEFFICIENTS OF FAILURE REGIONS OF 

version vm 





8.34 


8.35 


8.34 


8.42 


8.38 


8.34 


8.46 


8.46 


8.42 


8.42 


8.45 


8.46 


8.45 


8.5 

0 


8.3 


IV 




























6 






























8.3 


n 




























I 






























8.3 


u 


0 


n 
























6 






























8.3 


0 


0 


0.42 


n 






















8 






/ii 
























8.3 


0 


0 


0.4- 


1.0 


n 




















8 






3/II 


/ii 






















8.4 


0 


0 


0/11 


0/11 


0/11 


n 


. 


. 


. 


. 


m 


. 


. 




0 






























8.4 


0 


0 


0.5- 


0.75 


0.75 


0 


0 
















0 






0/11 


/II 


o/n 


/II 


















8.4 


0 


0 


0.44 


0.60 


0.60 


0/1 


0.83 


0 














1 






4/II 


/II 


/n 


I 


3/n 
















8.4 


0 


0 


0 


0 


0 


0 


0 


0/ 












_ 


4 
















V 














8.4 


0 


0 


0.36 


0.37 


0.37 


0 


0.44 


0.5- 


0/ 


• 










4 






4 


5 


5 




4 


0/V 


V 
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